OpenRAG is a comprehensive, single package Retrieval-Augmented Generation platform built on Langflow, Docling, and Opensearch.
Python
feat: Make models configurable,enable Azure AI Foundry and Azure OpenAI for Saas and CPD (#2293)
Edwin Jose8d036c12b97d
diff --git a/.env.example b/.env.exampleindex 22983b96c..e2e99365c 100644--- a/.env.example+++ b/.env.example@@ -588,6 +588,13 @@ INSTANA_SECRETS=regex:.*key.*,.*pass.*,.*secret.*,.*token.*,q\Z,search\Z,filenam# - "on_prem": Private CloudOPENRAG_RUN_MODE=oss+# OPTIONAL: Path to a YAML file listing which model providers are visible in+# each run mode above. Defaults to the shipped src/config/model_providers.yaml,+# which is what Settings, Onboarding and the model pickers all read. Set this+# only to override the shipped list for a deployment; an unreadable file falls+# back to the shipped one.+# OPENRAG_MODEL_PROVIDERS_CONFIG=+# Feature flag for the preview-mode ingest backend (default: false = disabled).# When true, the frontend can request preview ingests that show progress and a# live index proof without persisting Docling layout JSON. AND-ed with the rundiff --git a/MANIFEST.in b/MANIFEST.inindex d0f089fcb..57a8c9a09 100644--- a/MANIFEST.in+++ b/MANIFEST.in@@ -1 +1,2 @@-recursive-include src/tui/_assets *\ No newline at end of file+recursive-include src/tui/_assets *+recursive-include src/config *.yamldiff --git a/docs/docs/reference/configuration.mdx b/docs/docs/reference/configuration.mdxindex b682b004d..0f7134ca8 100644--- a/docs/docs/reference/configuration.mdx+++ b/docs/docs/reference/configuration.mdx@@ -63,6 +63,132 @@ These variables are initially set during the [application onboarding process](/i| `WATSONX_ENDPOINT` | Not set | Custom provider endpoint for the IBM watsonx.ai model provider. || `WATSONX_PROJECT_ID` | Not set | Project ID for the IBM watsonx.ai model provider. |+### Provider visibility per run mode {#model-provider-visibility}++Which providers appear in **Settings**, in onboarding, and in the model pickers+is not hardcoded. `src/config/model_providers.yaml` lists each provider and+whether it is shown in each run mode, and the backend filters that list by+`OPENRAG_RUN_MODE` (`oss`, `on_prem`, or `saas`). The UI renders whatever the+API returns, so there is no separate frontend list to keep in step.++```yaml+providers:+ - name: ollama+ display_name: Ollama+ modes:+ oss: true+ on_prem: true+ saas: false # local inference is not offered in SaaS+```++A mode key that is missing means the provider is hidden, so adding a run mode+never exposes a provider by accident. A provider the file does not list is not+offered in the UI at all, though a model id for it can still be typed by hand if+the gateway already routes it.++The shipped defaults are:++| Provider | `oss` | `on_prem` | `saas` |+|----------|-------|-----------|--------|+| `openai` | Yes | Yes | Yes |+| `ollama` | Yes | Yes | No |+| `watsonx` | Yes | Yes | Yes |+| `anthropic` | Yes | Yes | Yes |+| `azure_ai` | Yes | Yes | Yes |+| `azure` | Yes | Yes | Yes |++LiteLLM keys Azure twice, and the two are not interchangeable: `azure_ai` is+the Azure AI Foundry catalogue (Llama, Phi, Mistral, Cohere, and the gpt-5.x+entries Foundry serves), while `azure` is Azure OpenAI Service — the only one+of the two that carries the gpt-4.1 family. Offer both unless a deployment+uses just one.++To change what a deployment offers, edit the `modes` values and restart, or+point `OPENRAG_MODEL_PROVIDERS_CONFIG` at your own YAML file. The file is read+once per process, so the restart is what applies the edit; reload the console+afterwards to pick up the new list.++| Variable | Default | Description |+|----------|---------|-------------|+| `OPENRAG_MODEL_PROVIDERS_CONFIG` | Not set | Path to a YAML file that replaces the shipped `model_providers.yaml`. If the file cannot be read, the shipped list is used. |++#### Hide specific models {#exclude-models}++A row can also keep ids out of the pickers. `*` and `?` wildcards work, and+matching is against the id as the picker shows it — `gpt-3.5-turbo`, not+`openai/gpt-3.5-turbo`. The id's last path segment is matched too, so a plain+name also covers Azure's regional listings (`eu/gpt-5.6-luna`,+`us/gpt-5.6-luna`); write `eu/*` to reach one region on its own.++Exclusions are per provider row. Azure OpenAI serves the same models as+OpenAI, so a model you want gone from both needs the pattern on both rows.++```yaml+ - name: openai+ display_name: OpenAI+ modes:+ oss: true+ exclude_models:+ - gpt-3.5-* # a whole generation+ - gpt-5.6-luna # one model+```++Exclusion is applied last, so it also overrides a `models` entry on the same+provider, and it covers the live `/models/{provider}` lists as well as the+catalogue — otherwise an id hidden in Settings would reappear in onboarding.++Two other mechanisms already run without configuration, so reach for+`exclude_models` only when neither covers you:++| Situation | Handled by |+|---|---|+| The provider has already retired the model | Dropped from the catalogue automatically |+| The provider has announced a retirement within 90 days | Hidden behind a **Show N retiring** row in the picker |+| This deployment does not want the model offered at all | `exclude_models` |++Credentials for a provider without a purpose-built form — Azure AI Foundry+today — are entered in **Settings** through a form built from the provider's own+credential fields. Configuring one needs no code change beyond its row in this+file.++#### Add a custom OpenAI-compatible provider {#custom-openai-compatible-provider}++To offer a self-hosted OpenAI-compatible endpoint — vLLM, LM Studio, or an+internal gateway — add a row that declares the model ids it serves. The public+vendors need no `models` list because LiteLLM already ships one, but a private+gateway serves whatever its operator deployed, so those ids can only come from+this file.++```yaml+ - name: openai_like+ display_name: Internal LLM Gateway+ modes:+ oss: true+ on_prem: true+ saas: false+ models:+ - llama-3.3-70b-instruct+ - qwen2.5-coder-32b+ embedding_models:+ - bge-m3+```++Restart, then open **Settings** and configure the new card with the gateway's+**API base** (for example `https://gateway.internal/v1`) and API key. The+declared models then appear in the agent and ingestion pickers, and are served+from `/v1/models` as `openai_like:llama-3.3-70b-instruct`.++Use one of LiteLLM's passthrough provider keys for `name` — `openai_like`,+`hosted_vllm`, `lm_studio`, `llamafile`, or `custom_openai` — rather than+inventing one. The key must be a provider LiteLLM can route, or the+`provider:model` ids OpenRAG publishes resolve to the default provider and are+called with the wrong credentials. A name that is not routable is logged as a+warning when the catalogue is built.++Declaring `models` on a public vendor row also works, and is how you surface a+fine-tune or a private deployment that is not in LiteLLM's table. Ids that the+table already lists are not duplicated.+## Document processing settings {#document-processing-settings}Controls some aspects of how OpenRAG [processes and ingests documents](/ingestion) into your knowledge base.diff --git a/frontend/app/api/queries/useGetModelProvidersQuery.ts b/frontend/app/api/queries/useGetModelProvidersQuery.tsnew file mode 100644index 000000000..8980fe2f3--- /dev/null+++ b/frontend/app/api/queries/useGetModelProvidersQuery.ts@@ -0,0 +1,54 @@+import {+ type UseQueryOptions,+ useQuery,+ useQueryClient,+} from "@tanstack/react-query";++export interface ModelProviderEntry {+ name: string;+ display_name: string;+}++export interface ModelProvidersResponse {+ run_mode: string;+ providers: ModelProviderEntry[];+}++/**+ * The providers this deployment offers, filtered by OPENRAG_RUN_MODE.+ *+ * `config/model_providers.yaml` on the backend is the only source of truth for+ * this list: Settings cards, the Onboarding tabs and the model pickers all+ * render what it returns, so the frontend keeps no denylist of its own and+ * brand/theme never decides whether a provider is available.+ */+export const useGetModelProvidersQuery = (+ options?: Omit<+ UseQueryOptions<ModelProvidersResponse>,+ "queryKey" | "queryFn"+ >,+) => {+ const queryClient = useQueryClient();++ return useQuery(+ {+ queryKey: ["models", "providers"] as const,+ queryFn: async (): Promise<ModelProvidersResponse> => {+ const response = await fetch("/api/models/providers");+ if (!response.ok) {+ throw new Error("Failed to fetch the model providers");+ }+ return (await response.json()) as ModelProvidersResponse;+ },+ // Run mode and the config file are fixed for the life of the backend+ // process, so this never goes stale within a session. Editing+ // `config/model_providers.yaml` therefore takes a backend restart *and* a+ // page reload; the endpoint sends `no-store` so the reload actually asks.+ staleTime: Number.POSITIVE_INFINITY,+ gcTime: Number.POSITIVE_INFINITY,+ refetchOnWindowFocus: false,+ ...options,+ },+ queryClient,+ );+};diff --git a/frontend/app/chat/_components/message.tsx b/frontend/app/chat/_components/message.tsxindex b4985f4b7..c526d98ea 100644--- a/frontend/app/chat/_components/message.tsx+++ b/frontend/app/chat/_components/message.tsx@@ -16,13 +16,13 @@ export function Message({unstyledContent = false,}: MessageProps) {return (- <div className="flex gap-3">+ <div className="flex gap-3 overflow-hidden">{icon}<divclassName={isAssistant && !unstyledContent? "px-5 py-4 bg-secondary/20 rounded-2xl flex-1"- : "flex-1"+ : "flex-1 flex w-full overflow-hidden"}><div className="flex-1 min-w-0">{children}</div>diff --git a/frontend/app/onboarding/_components/generic-onboarding.tsx b/frontend/app/onboarding/_components/generic-onboarding.tsxnew file mode 100644index 000000000..493fea84d--- /dev/null+++ b/frontend/app/onboarding/_components/generic-onboarding.tsx@@ -0,0 +1,172 @@+import type { Dispatch, SetStateAction } from "react";+import { useMemo, useState } from "react";+import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";+import {+ onboardingCredentialFields,+ type SavedProvidersSnapshot,+ savedCredentialValuesForProvider,+ savedSecretFieldsForProvider,+} from "@/app/settings/_helpers/catalog-models";+import { getProviderChrome } from "@/app/settings/_helpers/model-helpers";+import { LabelInput } from "@/components/label-input";+import type { OnboardingVariables } from "../../api/mutations/useOnboardingMutation";+import { AdvancedOnboarding } from "./advanced";++/**+ * Onboarding step for a provider with no hand-built component.+ *+ * The credential inputs come from the catalogue's field spec and the model list+ * from the catalogue itself, so a provider added to+ * `config/model_providers.yaml` gets a working onboarding tab with no frontend+ * change. Credentials are submitted through the generic `provider_credentials`+ * payload the onboarding endpoint already accepts.+ */+export function GenericOnboarding({+ provider,+ displayName,+ setSettings,+ isEmbedding = false,+ providers,+}: {+ provider: string;+ displayName?: string;+ setSettings: Dispatch<SetStateAction<OnboardingVariables>>;+ isEmbedding?: boolean;+ providers?: SavedProvidersSnapshot;+}) {+ const { data: catalog } = useGetModelCatalogQuery();+ const chrome = getProviderChrome(provider, displayName);+ const Logo = chrome.logo;++ const fields = useMemo(+ () => onboardingCredentialFields(catalog, provider),+ [catalog, provider],+ );+ const savedSecrets = useMemo(+ () => new Set(savedSecretFieldsForProvider(providers, provider)),+ [providers, provider],+ );+ const savedValues = useMemo(+ () => savedCredentialValuesForProvider(providers, provider),+ [providers, provider],+ );++ const [credentials, setCredentials] = useState<Record<string, string>>({});+ const [model, setModel] = useState("");++ const syncParentSettings = (+ nextCredentials: Record<string, string>,+ nextModel: string,+ ) => {+ const submitted: Record<string, string> = {};+ for (const [key, value] of Object.entries(nextCredentials)) {+ const trimmed = (value ?? "").trim();+ if (trimmed !== "") {+ submitted[key] = trimmed;+ }+ }++ setSettings((prev) => ({+ ...prev,+ ...(isEmbedding+ ? { embedding_provider: provider, embedding_model: nextModel }+ : { llm_provider: provider, llm_model: nextModel }),+ provider_credentials: Object.keys(submitted).length+ ? { ...prev.provider_credentials, [provider]: submitted }+ : prev.provider_credentials,+ }));+ };++ // Seed the non-secret fields from what is already saved, once per provider.+ const [seededFor, setSeededFor] = useState<string | undefined>();+ if (seededFor !== provider) {+ setSeededFor(provider);+ setCredentials(savedValues);+ syncParentSettings(savedValues, model);+ }++ const catalogEntry = catalog?.providers?.find(+ (entry) => entry.key === provider,+ );+ const models = useMemo(() => {+ const entries = isEmbedding+ ? (catalogEntry?.embedding_models ?? [])+ : (catalogEntry?.models ?? []);+ return entries.map((entry) => ({+ value: entry.model,+ label: entry.model,+ }));+ }, [catalogEntry, isEmbedding]);++ // Default to the first model the catalogue lists for this provider.+ const [prevModels, setPrevModels] = useState<typeof models | undefined>();+ if (models !== prevModels) {+ setPrevModels(models);+ if (!model && models.length > 0) {+ const defaultModel = models[0].value;+ setModel(defaultModel);+ syncParentSettings(credentials, defaultModel);+ }+ }++ const handleCredentialChange = (fieldKey: string, newValue: string) => {+ const nextCredentials = { ...credentials, [fieldKey]: newValue };+ setCredentials(nextCredentials);+ syncParentSettings(nextCredentials, model);+ };++ const handleModelChange = (newModel: string) => {+ setModel(newModel);+ syncParentSettings(credentials, newModel);+ };++ return (+ <>+ <div className="space-y-5">+ {fields.map((field) => {+ const isSecret =+ field.field_type === "password" || field.field_type === "textarea";+ const hasSaved = isSecret && savedSecrets.has(field.key);+ return (+ <div key={field.key} className="space-y-1">+ <LabelInput+ label={field.label}+ helperText={field.tooltip ?? ""}+ id={`onboarding-${provider}-${field.key}`}+ type={field.field_type === "password" ? "password" : "text"}+ required={field.required && !hasSaved}+ placeholder={+ hasSaved ? "•••••••••" : (field.placeholder ?? undefined)+ }+ value={credentials[field.key] ?? ""}+ onChange={(e) =>+ handleCredentialChange(field.key, e.target.value)+ }+ />+ {hasSaved && (+ <p className="text-mmd text-muted-foreground">+ A value is already saved. Leave this blank to keep it.+ </p>+ )}+ </div>+ );+ })}+ {models.length === 0 && (+ <p className="text-mmd text-muted-foreground">+ {chrome.name} publishes no {isEmbedding ? "embedding" : "language"}{" "}+ models in the catalogue. Pick a different provider for this step.+ </p>+ )}+ </div>+ <AdvancedOnboarding+ icon={<Logo className="w-4 h-4" />}+ languageModels={isEmbedding ? undefined : models}+ embeddingModels={isEmbedding ? models : undefined}+ languageModel={isEmbedding ? undefined : model}+ embeddingModel={isEmbedding ? model : undefined}+ setLanguageModel={isEmbedding ? undefined : handleModelChange}+ setEmbeddingModel={isEmbedding ? handleModelChange : undefined}+ />+ </>+ );+}diff --git a/frontend/app/onboarding/_components/model-features.tsx b/frontend/app/onboarding/_components/model-features.tsxindex 7f3ed3f59..e0b7d014d 100644--- a/frontend/app/onboarding/_components/model-features.tsx+++ b/frontend/app/onboarding/_components/model-features.tsx@@ -23,7 +23,9 @@ import {PRIMARY_CAPABILITIES,supports,} from "@/app/settings/_helpers/model-info";+import AiFoundryLogo from "@/components/icons/ai-foundry-logo";import AnthropicLogo from "@/components/icons/anthropic-logo";+import AzureOpenAILogo from "@/components/icons/azure-openai-logo";import IBMLogo from "@/components/icons/ibm-logo";import OllamaLogo from "@/components/icons/ollama-logo";import OpenAILogo from "@/components/icons/openai-logo";@@ -95,6 +97,10 @@ const PROVIDER_LOGOS: Record<anthropic: AnthropicLogo,watsonx: IBMLogo,ollama: OllamaLogo,+ // Both Azure rows: `azure_ai` is Foundry, `azure` is Azure OpenAI Service,+ // and Microsoft draws the two differently.+ azure_ai: AiFoundryLogo,+ azure: AzureOpenAILogo,};function ProviderMark({ provider, name }: { provider?: string; name: string }) {diff --git a/frontend/app/onboarding/_components/model-selector.tsx b/frontend/app/onboarding/_components/model-selector.tsxindex af5905acb..891299127 100644--- a/frontend/app/onboarding/_components/model-selector.tsx+++ b/frontend/app/onboarding/_components/model-selector.tsx@@ -61,6 +61,31 @@ function optionProvider(return option.provider ?? group?.provider;}+/** How close a retirement has to be before the model drops out of the list. */+const RETIRING_SOON_DAYS = 90;++/**+ * A model close enough to retirement to be worth hiding by default.+ *+ * Not simply "has a deprecation_date": providers use that field very+ * differently. Anthropic stamps one on *every* model at launch, roughly a year+ * out, so hiding on presence alone would leave 2 of its 17 models on screen.+ * OpenAI leaves current models undated and sunsets a whole legacy generation+ * on one shared date. A window reads correctly for both — it hides OpenAI's+ * gpt-3.5 family and Anthropic's 4-5 generation, and nothing at all for+ * providers that publish no dates.+ *+ * The backend already drops models whose date has passed, so anything here is+ * still callable — hidden, never taken away.+ */+function isRetiringSoon(option: ModelOption): boolean {+ const date = option.model?.deprecation_date;+ if (!date) return false;+ const retires = Date.parse(date);+ if (Number.isNaN(retires)) return false;+ return retires <= Date.now() + RETIRING_SOON_DAYS * 86_400_000;+}+function isSelectedRow(option: ModelOption,value: string,@@ -85,7 +110,7 @@ export function ModelSelector({onValueChange,icon,placeholder = "Select model...",- searchPlaceholder = "Search model...",+ searchPlaceholder,noOptionsPlaceholder = "No models available",custom = false,hasError = false,@@ -104,6 +129,14 @@ export function ModelSelector({}}+ // A selector that accepts a typed model has to say so: the box is the only+ // way to reach a deployment name the catalogue cannot know. Selectors that+ // only pick from a fixed list (the watsonx endpoint picker) keep the plain+ // label, and any caller passing its own copy still wins.+ const searchBoxPlaceholder =+ searchPlaceholder ??+ (custom ? "Search or type a model name" : "Search model...");+const [searchValue, setSearchValue] = useState("");// The option filter runs on the trimmed, lowercased search text, so the// custom entry has to use the trimmed text too — otherwise typing trailing@@ -122,6 +155,16 @@ export function ModelSelector({if (!next.delete(group)) next.add(group);return next;});+ // Groups the user chose to see retiring models in.+ const [deprecatedGroups, setDeprecatedGroups] = useState<Set<string>>(+ () => new Set(),+ );+ const toggleDeprecated = (group: string) =>+ setDeprecatedGroups((previous) => {+ const next = new Set(previous);+ if (!next.delete(group)) next.add(group);+ return next;+ });// Flatten grouped options or use regular optionsconst allOptions =@@ -145,24 +188,46 @@ export function ModelSelector({const providerMatches = group.group.toLowerCase().includes(deferredSearch);- const matches = deferredSearch+ const searched = deferredSearch? group.options.filter((option) =>providerMatches ||option.label.toLowerCase().includes(deferredSearch),): group.options;- if (deferredSearch && matches.length === 0) return [];+ // Retiring models are hidden until asked for, but never the one already+ // selected — taking the current choice off the list would make the+ // selector look broken to whoever set it.+ const deprecatedCount = searched.filter(isRetiringSoon).length;+ const matches = deprecatedGroups.has(group.group)+ ? searched+ : searched.filter(+ (option) =>+ !isRetiringSoon(option) ||+ isSelectedRow(option, value, selectedProvider, group),+ );+ if (deferredSearch && matches.length === 0 && deprecatedCount === 0) {+ return [];+ }// Collapsed groups show a preview; the trailing row expands them. The cap// applies while searching too, so without that row a search would drop// matches with nothing on screen to say so.const options = expandedGroups.has(group.group)? matches: matches.slice(0, MODELS_PER_PROVIDER);- return [{ ...group, options, matchCount: matches.length }];+ return [+ { ...group, options, matchCount: matches.length, deprecatedCount },+ ];});return matched.slice(0, 40);- }, [deferredSearch, groupedOptions, expandedGroups]);+ }, [+ deferredSearch,+ groupedOptions,+ expandedGroups,+ deprecatedGroups,+ value,+ selectedProvider,+ ]);const visibleOptions = useMemo(() => {if (groupedOptions) return [];const source = options ?? [];@@ -244,7 +309,10 @@ export function ModelSelector({><Command shouldFilter={false}><CommandInput- placeholder={searchPlaceholder}+ // Located by test id, not by placeholder: the copy changes with+ // `custom` and is the sort of thing that gets reworded.+ data-testid="model-search-input"+ placeholder={searchBoxPlaceholder}value={searchValue}onValueChange={setSearchValue}/>@@ -366,6 +434,27 @@ export function ModelSelector({: "Show fewer"}</CommandItem>)}+ {group.deprecatedCount > 0 && (+ <CommandItem+ value={`__deprecated-${group.group}`}+ aria-label={+ deprecatedGroups.has(group.group)+ ? `Hide retiring ${group.group} models`+ : `Show ${group.deprecatedCount} retiring ${group.group} models`+ }+ data-testid={+ groupProvider+ ? `model-group-deprecated-${groupProvider}`+ : undefined+ }+ className="text-xs text-muted-foreground"+ onSelect={() => toggleDeprecated(group.group)}+ >+ {deprecatedGroups.has(group.group)+ ? "Hide retiring models"+ : `Show ${group.deprecatedCount} retiring`}+ </CommandItem>+ )}{showCustom && (<CommandItemvalue={`${group.group}-${customValue}`}@@ -392,7 +481,23 @@ export function ModelSelector({)}/><div className="flex items-center gap-2">- {customValue}+ {/* "Use" rather than the bare name: the row sits+ among search results, and without a verb it+ reads as one more model that already exists+ rather than the thing that adds yours. */}+ <span>Use</span>+ {/* The provider tag is shown, never typed: the+ group the row sits under decides it, and it is+ what makes the id route back here. Without it+ on screen there is nothing to tell you a name+ typed under Azure OpenAI is stored as+ `azure:<name>`. */}+ {groupProvider && (+ <span className="rounded-sm bg-muted px-1 font-mono text-xs text-muted-foreground">+ {groupProvider}:+ </span>+ )}+ <span>“{customValue}”</span><span className="text-xs text-foreground p-1 rounded-md bg-muted">Custom</span>@@ -422,7 +527,7 @@ export function ModelSelector({)}/><div className="flex items-center gap-2">- {customValue}+ Use “{customValue}”<span className="text-xs text-foreground p-1 rounded-md bg-muted">Custom</span>@@ -484,7 +589,7 @@ export function ModelSelector({)}/><div className="flex items-center gap-2">- {customValue}+ Use “{customValue}”<span className="text-xs text-foreground p-1 rounded-md bg-muted">Custom</span>diff --git a/frontend/app/onboarding/_components/onboarding-card.tsx b/frontend/app/onboarding/_components/onboarding-card.tsxindex 67cb08904..49d660e40 100644--- a/frontend/app/onboarding/_components/onboarding-card.tsx+++ b/frontend/app/onboarding/_components/onboarding-card.tsx@@ -3,13 +3,15 @@import { useIsFetching, useQueryClient } from "@tanstack/react-query";import { AnimatePresence, motion } from "framer-motion";import { X } from "lucide-react";-import { useEffect, useEffectEvent, useRef, useState } from "react";+import { useEffect, useEffectEvent, useMemo, useRef, useState } from "react";import { toast } from "sonner";import {type OnboardingVariables,useOnboardingMutation,} from "@/app/api/mutations/useOnboardingMutation";import { useOnboardingRollbackMutation } from "@/app/api/mutations/useOnboardingRollbackMutation";+import { useGetModelProvidersQuery } from "@/app/api/queries/useGetModelProvidersQuery";+import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";import {type ProviderSettings,useGetSettingsQuery,@@ -17,15 +19,12 @@ import {import { useGetTasksQuery } from "@/app/api/queries/useGetTasksQuery";import type { ProviderHealthResponse } from "@/app/api/queries/useProviderHealthQuery";import {- CLOUD_EXCLUDED_PROVIDERS,EMBEDDING_PROVIDER_ORDER,+ getProviderChrome,LLM_PROVIDER_ORDER,+ orderProviders,} from "@/app/settings/_helpers/model-helpers";import { useDoclingHealth } from "@/components/docling-health-banner";-import AnthropicLogo from "@/components/icons/anthropic-logo";-import IBMLogo from "@/components/icons/ibm-logo";-import OllamaLogo from "@/components/icons/ollama-logo";-import OpenAILogo from "@/components/icons/openai-logo";import { Button } from "@/components/ui/button";import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";import {@@ -33,7 +32,6 @@ import {TooltipContent,TooltipTrigger,} from "@/components/ui/tooltip";-import { useIsCloudBrand } from "@/contexts/brand-context";import {trackButton,trackProcessFailure,@@ -43,6 +41,7 @@ import { formatProviderErrorMessage } from "@/lib/chat-stream-errors";import { cn } from "@/lib/utils";import { AnimatedProviderSteps } from "./animated-provider-steps";import { AnthropicOnboarding } from "./anthropic-onboarding";+import { GenericOnboarding } from "./generic-onboarding";import { IBMOnboarding } from "./ibm-onboarding";import { OllamaOnboarding } from "./ollama-onboarding";import { OpenAIOnboarding } from "./openai-onboarding";@@ -60,6 +59,10 @@ const STEP_LIST = ["Configuring Langflow",];+// Providers whose model inventory is read live from the running service, so an+// empty catalogue list says nothing about what they can serve.+const LIVE_MODEL_PROVIDERS = new Set(["ollama", "watsonx"]);+const EMBEDDING_STEP_LIST = ["Setting up your model provider","Defining schema",@@ -73,12 +76,70 @@ const OnboardingCard = ({isCompleted = false,}: OnboardingCardProps) => {const { isHealthy: isDoclingHealthy } = useDoclingHealth();- const isCloudBrand = useIsCloudBrand();++ // Which providers this deployment offers comes from the backend, filtered by+ // OPENRAG_RUN_MODE (config/model_providers.yaml). Onboarding renders that+ // list; it does not decide availability from the UI brand.+ const { data: providerData } = useGetModelProvidersQuery();+ const availableProviders = useMemo(+ () => providerData?.providers ?? [],+ [providerData],+ );+ const providerDisplayNames = useMemo(+ () =>+ Object.fromEntries(+ availableProviders.map(({ name, display_name }) => [+ name,+ display_name,+ ]),+ ),+ [availableProviders],+ );+ const providerKeys = useMemo(+ () =>+ orderProviders(+ availableProviders.map((provider) => provider.name),+ isEmbedding ? EMBEDDING_PROVIDER_ORDER : LLM_PROVIDER_ORDER,+ ),+ [availableProviders, isEmbedding],+ );++ const { data: catalog } = useGetModelCatalogQuery();++ // The embedding step must not offer a provider that serves no embedding+ // models — Anthropic being the standing example. The catalogue answers that+ // for every provider except the two whose inventory comes from the running+ // server rather than LiteLLM's bundled list.+ const tabProviders = useMemo(() => {+ if (!isEmbedding) {+ return providerKeys;+ }+ return providerKeys.filter((providerKey) => {+ if (LIVE_MODEL_PROVIDERS.has(providerKey)) {+ return true;+ }+ const entry = catalog?.providers?.find(+ (item) => item.key === providerKey,+ );+ // Catalogue not loaded yet: hide nothing rather than flicker tabs away.+ return !entry || entry.embedding_models.length > 0;+ });+ }, [providerKeys, isEmbedding, catalog]);const [modelProvider, setModelProvider] = useState<string>(isEmbedding ? "openai" : "anthropic",);+ // The default above may not be offered here (Anthropic disabled, or the+ // embedding step). Fall back to the first provider this step does offer.+ const [prevTabProviders, setPrevTabProviders] = useState<string[]>([]);+ if (tabProviders !== prevTabProviders && tabProviders.length > 0) {+ setPrevTabProviders(tabProviders);+ if (!tabProviders.includes(modelProvider)) {+ setModelProvider(tabProviders[0]);+ }+ }+// Read model-fetch loading from React Query instead of syncing it up from children.const isLoadingModels = useIsFetching({ queryKey: ["models"] }) > 0;@@ -94,14 +155,7 @@ const OnboardingCard = ({if (currentSettings?.providers !== prevProviders) {setPrevProviders(currentSettings?.providers);if (currentSettings?.providers) {- const fullOrder = isEmbedding- ? EMBEDDING_PROVIDER_ORDER- : LLM_PROVIDER_ORDER;- const providerOrder = isCloudBrand- ? fullOrder.filter((p) => !CLOUD_EXCLUDED_PROVIDERS.includes(p))- : fullOrder;-- for (const provider of providerOrder) {+ for (const provider of tabProviders) {if (provider === "anthropic" &¤tSettings.providers.anthropic?.has_api_key@@ -126,6 +180,11 @@ const OnboardingCard = ({) {setModelProvider("ollama");break;+ } else if (+ currentSettings.providers.custom?.[provider]?.configured === true+ ) {+ setModelProvider(provider);+ break;}}}@@ -155,7 +214,7 @@ const OnboardingCard = ({} else if (provider === "ollama") {return currentSettings.providers.ollama?.configured === true;}- return false;+ return currentSettings.providers.custom?.[provider]?.configured === true;};const showProviderConfiguredMessage =@@ -476,6 +535,13 @@ const OnboardingCard = ({onboardingData.ollama_endpoint = settings.ollama_endpoint;}+ // Providers configured through the generic form submit their credentials+ // as-is; the backend stores whatever field keys the provider declares.+ const generic = settings.provider_credentials?.[currentProvider];+ if (generic && Object.keys(generic).length > 0) {+ onboardingData.provider_credentials = { [currentProvider]: generic };+ }+trackButton({CTA: isEmbedding ? "Complete - Embedding Setup" : "Complete - LLM Setup",elementId: "onboarding-complete-button",@@ -508,7 +574,7 @@ const OnboardingCard = ({exit={{ opacity: 0, y: 24 }}transition={{ duration: 0.4, ease: "easeInOut" }}>- <div className={`w-full max-w-[600px] flex flex-col`}>+ <div className={`w-full flex flex-col`}><AnimatePresence mode="wait">{error && (<motion.div@@ -533,198 +599,119 @@ const OnboardingCard = ({value={modelProvider}onValueChange={handleSetModelProvider}>- <TabsList className="mb-4">- {!isEmbedding && (- <TabsTrigger- value="anthropic"- data-testid={`anthropic-llm-tab`}- className={cn(- error &&- modelProvider === "anthropic" &&- "data-[state=active]:border-destructive",- )}- >- <TabTrigger- selected={modelProvider === "anthropic"}- isLoading={isLoadingModels}- >- <div- className={cn(- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",- modelProvider === "anthropic"- ? "bg-[#D97757]"- : "bg-muted",- )}- >- <AnthropicLogo- className={cn(- "w-4 h-4 shrink-0",- modelProvider === "anthropic"- ? "text-black"- : "text-muted-foreground",- )}- />- </div>- Anthropic- </TabTrigger>- </TabsTrigger>- )}- <TabsTrigger- value="openai"- className={cn(- error &&- modelProvider === "openai" &&- "data-[state=active]:border-destructive",- )}- data-testid={`openai-${isEmbedding ? "embedding" : "llm"}-tab`}- >- <TabTrigger- selected={modelProvider === "openai"}- isLoading={isLoadingModels}- >- <div+ <TabsList className="mb-1 pb-3 w-full justify-start gap-1 overflow-x-auto">+ {tabProviders.map((providerKey) => {+ const chrome = getProviderChrome(+ providerKey,+ providerDisplayNames[providerKey],+ );+ const Logo = chrome.logo;+ const selected = modelProvider === providerKey;+ return (+ <TabsTrigger+ key={providerKey}+ value={providerKey}+ data-testid={`${providerKey}-${isEmbedding ? "embedding" : "llm"}-tab`}className={cn(- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",- modelProvider === "openai" ? "bg-white" : "bg-muted",+ error &&+ selected &&+ "data-[state=active]:border-destructive",+ "min-w-40",)}>- <OpenAILogo- className={cn(- "w-4 h-4 shrink-0",- modelProvider === "openai"- ? "text-black"- : "text-muted-foreground",- )}- />- </div>- OpenAI- </TabTrigger>- </TabsTrigger>- <TabsTrigger- value="watsonx"- data-testid={`watsonx-${isEmbedding ? "embedding" : "llm"}-tab`}- className={cn(- error &&- modelProvider === "watsonx" &&- "data-[state=active]:border-destructive",- )}- >- <TabTrigger- selected={modelProvider === "watsonx"}- isLoading={isLoadingModels}- >- <div- className={cn(- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",- modelProvider === "watsonx"- ? "bg-[#1063FE]"- : "bg-muted",- )}- >- <IBMLogo- className={cn(- "w-4 h-4 shrink-0",- modelProvider === "watsonx"- ? "text-white"- : "text-muted-foreground",- )}- />- </div>- IBM watsonx.ai- </TabTrigger>- </TabsTrigger>- {!isCloudBrand && (- <TabsTrigger- value="ollama"- data-testid={`ollama-${isEmbedding ? "embedding" : "llm"}-tab`}- className={cn(- error &&- modelProvider === "ollama" &&- "data-[state=active]:border-destructive",- )}- >- <TabTrigger- selected={modelProvider === "ollama"}- isLoading={isLoadingModels}- >- <div- className={cn(- "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",- modelProvider === "ollama"- ? "bg-white"- : "bg-muted",- )}+ <TabTrigger+ selected={selected}+ isLoading={isLoadingModels}>- <OllamaLogo+ <divclassName={cn(- "w-4 h-4 shrink-0",- modelProvider === "ollama"- ? "text-black"- : "text-muted-foreground",+ "flex items-center justify-center gap-2 w-8 h-8 rounded-none border",+ selected+ ? (chrome.tabLogoBgColor ?? chrome.logoBgColor)+ : "bg-muted",)}- />- </div>- Ollama- </TabTrigger>- </TabsTrigger>- )}+ >+ <Logo+ className={cn(+ "w-4 h-4 shrink-0",+ selected+ ? (chrome.tabLogoColor ?? chrome.logoColor)+ : "text-muted-foreground",+ )}+ />+ </div>+ {chrome.name}+ </TabTrigger>+ </TabsTrigger>+ );+ })}</TabsList>- {!isEmbedding && (- <TabsContent value="anthropic">- <AnthropicOnboarding- setSettings={setSettings}- isEmbedding={isEmbedding}- hasEnvApiKey={- currentSettings?.providers?.anthropic?.has_api_key ===- true- }- />- </TabsContent>- )}- <TabsContent value="openai">- <OpenAIOnboarding- setSettings={setSettings}- isEmbedding={isEmbedding}- hasEnvApiKey={- currentSettings?.providers?.openai?.has_api_key === true- }- alreadyConfigured={- providerAlreadyConfigured && modelProvider === "openai"- }- />- </TabsContent>- <TabsContent value="watsonx">- <IBMOnboarding- setSettings={setSettings}- isEmbedding={isEmbedding}- alreadyConfigured={- providerAlreadyConfigured && modelProvider === "watsonx"- }- existingEndpoint={- currentSettings?.providers?.watsonx?.endpoint- }- existingProjectId={- currentSettings?.providers?.watsonx?.project_id- }- hasEnvApiKey={- currentSettings?.providers?.watsonx?.has_api_key === true- }- />- </TabsContent>- {!isCloudBrand && (- <TabsContent value="ollama">- <OllamaOnboarding- setSettings={setSettings}- isEmbedding={isEmbedding}- alreadyConfigured={- providerAlreadyConfigured && modelProvider === "ollama"- }- existingEndpoint={- currentSettings?.providers?.ollama?.endpoint- }- />+ {tabProviders.map((providerKey) => (+ <TabsContent key={providerKey} value={providerKey}>+ {providerKey === "anthropic" ? (+ <AnthropicOnboarding+ setSettings={setSettings}+ isEmbedding={isEmbedding}+ hasEnvApiKey={+ currentSettings?.providers?.anthropic?.has_api_key ===+ true+ }+ />+ ) : providerKey === "openai" ? (+ <OpenAIOnboarding+ setSettings={setSettings}+ isEmbedding={isEmbedding}+ hasEnvApiKey={+ currentSettings?.providers?.openai?.has_api_key ===+ true+ }+ alreadyConfigured={+ providerAlreadyConfigured &&+ modelProvider === "openai"+ }+ />+ ) : providerKey === "watsonx" ? (+ <IBMOnboarding+ setSettings={setSettings}+ isEmbedding={isEmbedding}+ alreadyConfigured={+ providerAlreadyConfigured &&+ modelProvider === "watsonx"+ }+ existingEndpoint={+ currentSettings?.providers?.watsonx?.endpoint+ }+ existingProjectId={+ currentSettings?.providers?.watsonx?.project_id+ }+ hasEnvApiKey={+ currentSettings?.providers?.watsonx?.has_api_key ===+ true+ }+ />+ ) : providerKey === "ollama" ? (+ <OllamaOnboarding+ setSettings={setSettings}+ isEmbedding={isEmbedding}+ alreadyConfigured={+ providerAlreadyConfigured &&+ modelProvider === "ollama"+ }+ existingEndpoint={+ currentSettings?.providers?.ollama?.endpoint+ }+ />+ ) : (+ <GenericOnboarding+ provider={providerKey}+ displayName={providerDisplayNames[providerKey]}+ setSettings={setSettings}+ isEmbedding={isEmbedding}+ providers={currentSettings?.providers}+ />+ )}</TabsContent>- )}+ ))}</Tabs><Tooltip>diff --git a/frontend/app/onboarding/_components/onboarding-step.tsx b/frontend/app/onboarding/_components/onboarding-step.tsxindex b2941f6b4..bf9cc2bbe 100644--- a/frontend/app/onboarding/_components/onboarding-step.tsx+++ b/frontend/app/onboarding/_components/onboarding-step.tsx@@ -84,7 +84,7 @@ export function OnboardingStep({initial={{ opacity: 0, y: -20 }}animate={{ opacity: 1, y: 0 }}transition={{ duration: 0.4, delay: 0.4, ease: "easeOut" }}- className={isCompleted ? "opacity-50" : ""}+ className={cn(isCompleted ? "opacity-50" : "", "pr-10 mr-1 flex w-full")}><Messageicon={diff --git a/frontend/app/settings/_components/ingest-settings-section.tsx b/frontend/app/settings/_components/ingest-settings-section.tsxindex 08aa61fb6..39a9ca5f2 100644--- a/frontend/app/settings/_components/ingest-settings-section.tsx+++ b/frontend/app/settings/_components/ingest-settings-section.tsx@@ -3,7 +3,11 @@import { ArrowUpRight, ChevronDown, Loader2, Minus, Plus } from "lucide-react";import { useCallback, useEffect, useMemo, useRef, useState } from "react";import { toast } from "sonner";-import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";+import {+ useGetIBMModelsQuery,+ useGetModelCatalogQuery,+ useGetOllamaModelsQuery,+} from "@/app/api/queries/useGetModelsQuery";import { useGetSettingsQuery } from "@/app/api/queries/useGetSettingsQuery";import { ConfirmationDialog } from "@/components/confirmation-dialog";import { LabelWrapper } from "@/components/label-wrapper";@@ -49,6 +53,9 @@ import {import {findGroupedSelection,groupedCatalogOptions,+ LIVE_INVENTORY_PROVIDERS,+ liveModelOption,+ mergeLiveCatalogOptions,} from "../_helpers/catalog-models";import { getModelLogo } from "../_helpers/model-helpers";import { LangflowIcon } from "./langflow-icon";@@ -125,20 +132,71 @@ export function IngestSettingsSection() {[settings.providers],);- const groupedEmbeddingModels = useMemo(- () =>- groupedCatalogOptions(catalog, configuredProviders, "embedding").map(- (group) => ({- group: group.group,- provider: group.key,- icon: getModelLogo("", group.key),- options: group.options,- }),- ),- [catalog, configuredProviders],+ // watsonx and Ollama publish their inventory from the running server, not+ // from LiteLLM's bundled table, which carries *no* embedding models for+ // either of them — so the live list is the only thing that puts their group+ // in this picker at all.+ //+ // Fetched whenever the provider is configured, never gated on it already+ // being the selected embedding provider: the whole point of the group is to+ // be there before you pick it. Gating on selection is why watsonx stayed+ // missing after the first attempt at this.+ const authed = isAuthenticated || isNoAuthMode;+ const watsonx = settings.providers?.watsonx;+ const ollama = settings.providers?.ollama;++ const ibmModels = useGetIBMModelsQuery(+ {+ endpoint: watsonx?.endpoint,+ projectId: watsonx?.project_id,+ useEnvKey: true,+ },+ {+ enabled:+ authed &&+ configuredProviders.watsonx &&+ !!watsonx?.endpoint &&+ !!watsonx?.project_id,+ },+ );+ const ollamaModels = useGetOllamaModelsQuery(+ { endpoint: ollama?.endpoint },+ { enabled: authed && configuredProviders.ollama && !!ollama?.endpoint },);- const isLoadingAnyEmbeddingModels = catalogLoading;+ const groupedEmbeddingModels = useMemo(() => {+ const named = (key: string) =>+ catalog?.providers?.find((provider) => provider.key === key)?.name;+ const liveFor = (key: string, models?: { value: string }[]) =>+ (models ?? []).map((model) => liveModelOption(model.value, key));++ let groups = groupedCatalogOptions(+ catalog,+ configuredProviders,+ "embedding",+ );+ const liveByProvider: Record<string, { value: string }[] | undefined> = {+ watsonx: ibmModels.data?.embedding_models,+ ollama: ollamaModels.data?.embedding_models,+ };+ for (const key of LIVE_INVENTORY_PROVIDERS) {+ groups = mergeLiveCatalogOptions(+ groups,+ key,+ liveFor(key, liveByProvider[key]),+ named(key),+ );+ }+ return groups.map((group) => ({+ group: group.group,+ provider: group.key,+ icon: getModelLogo("", group.key),+ options: group.options,+ }));+ }, [catalog, configuredProviders, ibmModels.data, ollamaModels.data]);++ const isLoadingAnyEmbeddingModels =+ catalogLoading || ibmModels.isLoading || ollamaModels.isLoading;const groupedVlmModels = useMemo(() => {const list: GroupedModelOption[] = groupedCatalogOptions(diff --git a/frontend/app/settings/_components/model-providers.tsx b/frontend/app/settings/_components/model-providers.tsxindex b5cc970de..8fcf06c5b 100644--- a/frontend/app/settings/_components/model-providers.tsx+++ b/frontend/app/settings/_components/model-providers.tsx@@ -1,51 +1,69 @@"use client";import { useRouter, useSearchParams } from "next/navigation";-import { type ReactNode, useEffect, useMemo, useState } from "react";+import { useEffect, useMemo, useState } from "react";+import { useGetModelProvidersQuery } from "@/app/api/queries/useGetModelProvidersQuery";import { useGetSettingsQuery } from "@/app/api/queries/useGetSettingsQuery";-import AnthropicLogo from "@/components/icons/anthropic-logo";-import IBMLogo from "@/components/icons/ibm-logo";-import OllamaLogo from "@/components/icons/ollama-logo";-import OpenAILogo from "@/components/icons/openai-logo";import { useProviderHealth } from "@/components/provider-health-banner";import { useAuth } from "@/contexts/auth-context";-import { useIsCloudBrand } from "@/contexts/brand-context";import {- ALL_PROVIDERS,- CLOUD_EXCLUDED_PROVIDERS,+ getProviderChrome,type ModelProvider,} from "../_helpers/model-helpers";import AnthropicSettingsDialog from "./anthropic-settings-dialog";import ModelProviderCard from "./model-provider-card";import OllamaSettingsDialog from "./ollama-settings-dialog";import OpenAISettingsDialog from "./openai-settings-dialog";+import ProviderSettingsDialog from "./provider-settings-dialog";import WatsonxSettingsDialog from "./watsonx-settings-dialog";+// Providers with a hand-built credential dialog. Everything else the backend+// offers is configured through the catalogue-driven ProviderSettingsDialog.+const BESPOKE_DIALOG_PROVIDERS = new Set<ModelProvider>([+ "openai",+ "anthropic",+ "ollama",+ "watsonx",+]);+export const ModelProviders = () => {const { isAuthenticated, isNoAuthMode } = useAuth();const searchParams = useSearchParams();const router = useRouter();- const isCloudBrand = useIsCloudBrand();const { data: settings = {} } = useGetSettingsQuery({enabled: isAuthenticated || isNoAuthMode,});+ // The backend filters this list by OPENRAG_RUN_MODE (see+ // config/model_providers.yaml). The UI keeps no denylist of its own, and the+ // IBM theme no longer decides what is available.+ const {+ data: providerData,+ isLoading: isLoadingProviders,+ isError: providersFailed,+ } = useGetModelProvidersQuery({+ enabled: isAuthenticated || isNoAuthMode,+ });+const { health } = useProviderHealth();const [dialogOpen, setDialogOpen] = useState<ModelProvider | undefined>();- const allProviderKeys = useMemo(() => {- return isCloudBrand- ? ALL_PROVIDERS.filter((p) => !CLOUD_EXCLUDED_PROVIDERS.includes(p))- : ALL_PROVIDERS;- }, [isCloudBrand]);+ const providers = useMemo(+ () => providerData?.providers ?? [],+ [providerData],+ );+ const allProviderKeys = useMemo(+ () => providers.map((provider) => provider.name),+ [providers],+ );// Handle URL search param to open dialogsuseEffect(() => {const searchParam = searchParams.get("setup");- if (searchParam && allProviderKeys.includes(searchParam as ModelProvider)) {- setDialogOpen(searchParam as ModelProvider);+ if (searchParam && allProviderKeys.includes(searchParam)) {+ setDialogOpen(searchParam);}}, [searchParams, allProviderKeys]);@@ -61,78 +79,28 @@ export const ModelProviders = () => {router.replace(newUrl);};- const modelProvidersMap: Record<- ModelProvider,- {- name: string;- logo: (props: React.SVGProps<SVGSVGElement>) => ReactNode;- logoColor: string;- logoBgColor: string;- }- > = {- openai: {- name: "OpenAI",- logo: OpenAILogo,- logoColor: "text-black",- logoBgColor: "bg-white",- },- anthropic: {- name: "Anthropic",- logo: AnthropicLogo,- logoColor: "text-[#D97757]",- logoBgColor: "bg-white",- },- ollama: {- name: "Ollama",- logo: OllamaLogo,- logoColor: "text-black",- logoBgColor: "bg-white",- },- watsonx: {- name: "IBM watsonx.ai",- logo: IBMLogo,- logoColor: "text-white",- logoBgColor: "bg-[#1063FE]",- },- local: {- name: "Local",- logo: (props) => (- <svg- xmlns="http://www.w3.org/2000/svg"- viewBox="0 0 24 24"- fill="none"- stroke="currentColor"- strokeWidth="2"- strokeLinecap="round"- strokeLinejoin="round"- {...props}- >- <rect x="4" y="4" width="16" height="16" rx="2" />- <rect x="9" y="9" width="6" height="6" />- <path d="M9 1v3" />- <path d="M15 1v3" />- <path d="M9 20v3" />- <path d="M15 20v3" />- <path d="M20 9h3" />- <path d="M20 15h3" />- <path d="M1 9h3" />- <path d="M1 15h3" />- </svg>- ),- logoColor: "text-muted-foreground",- logoBgColor: "bg-white",- },- };-const currentLlmProvider =(settings.agent?.llm_provider as ModelProvider) || "openai";const currentEmbeddingProvider =(settings.knowledge?.embedding_provider as ModelProvider) || "openai";+ const genericDialogProvider =+ dialogOpen && !BESPOKE_DIALOG_PROVIDERS.has(dialogOpen)+ ? dialogOpen+ : undefined;++ if (providersFailed) {+ return (+ <p className="text-sm text-destructive">+ The list of model providers could not be loaded. Refresh to try again.+ </p>+ );+ }+return (<><div className="grid gap-6 xs:grid-cols-1 md:grid-cols-2 lg:grid-cols-4">- {allProviderKeys.map((providerKey) => {+ {providers.map(({ name: providerKey, display_name }) => {const isLlmProvider = providerKey === currentLlmProvider;const isEmbeddingProvider = providerKey === currentEmbeddingProvider;const isProviderUnhealthy =@@ -142,30 +110,62 @@ export const ModelProviders = () => {return (<ModelProviderCardkey={providerKey}- provider={{ providerKey, ...modelProvidersMap[providerKey] }}- isConfigured={!!settings.providers?.[providerKey]?.configured}+ provider={{+ providerKey,+ ...getProviderChrome(providerKey, display_name),+ }}+ // `providers.custom` carries every provider the backend knows,+ // legacy four included, so one lookup covers config-added ones.+ isConfigured={+ settings.providers?.custom?.[providerKey]?.configured === true+ }isUnhealthy={!!isProviderUnhealthy}onConfigure={setDialogOpen}/>);})}</div>- <AnthropicSettingsDialog- open={dialogOpen === "anthropic"}- setOpen={handleCloseDialog}- />- <OpenAISettingsDialog- open={dialogOpen === "openai"}- setOpen={handleCloseDialog}- />- <OllamaSettingsDialog- open={dialogOpen === "ollama"}- setOpen={handleCloseDialog}- />- <WatsonxSettingsDialog- open={dialogOpen === "watsonx"}- setOpen={handleCloseDialog}- />+ {!isLoadingProviders && providers.length === 0 && (+ <p className="text-sm text-muted-foreground">+ No model providers are enabled for this deployment.+ </p>+ )}+ {allProviderKeys.includes("anthropic") && (+ <AnthropicSettingsDialog+ open={dialogOpen === "anthropic"}+ setOpen={handleCloseDialog}+ />+ )}+ {allProviderKeys.includes("openai") && (+ <OpenAISettingsDialog+ open={dialogOpen === "openai"}+ setOpen={handleCloseDialog}+ />+ )}+ {allProviderKeys.includes("ollama") && (+ <OllamaSettingsDialog+ open={dialogOpen === "ollama"}+ setOpen={handleCloseDialog}+ />+ )}+ {allProviderKeys.includes("watsonx") && (+ <WatsonxSettingsDialog+ open={dialogOpen === "watsonx"}+ setOpen={handleCloseDialog}+ />+ )}+ {genericDialogProvider && (+ <ProviderSettingsDialog+ provider={genericDialogProvider}+ displayName={+ providers.find(+ (provider) => provider.name === genericDialogProvider,+ )?.display_name+ }+ open+ setOpen={handleCloseDialog}+ />+ )}</>);};diff --git a/frontend/app/settings/_components/provider-settings-dialog.tsx b/frontend/app/settings/_components/provider-settings-dialog.tsxnew file mode 100644index 000000000..aaa901b00--- /dev/null+++ b/frontend/app/settings/_components/provider-settings-dialog.tsx@@ -0,0 +1,259 @@+"use client";++import { AnimatePresence, domAnimation, LazyMotion, m } from "motion/react";+import { useRouter } from "next/navigation";+import { useEffect, useState } from "react";+import { FormProvider, useForm } from "react-hook-form";+import { toast } from "sonner";+import {+ type AffectedEmbeddingModel,+ isEmbeddingProviderInUseError,+ useUpdateSettingsMutation,+} from "@/app/api/mutations/useUpdateSettingsMutation";+import { useGetModelCatalogQuery } from "@/app/api/queries/useGetModelsQuery";+import { useGetSettingsQuery } from "@/app/api/queries/useGetSettingsQuery";+import {+ Dialog,+ DialogContent,+ DialogHeader,+ DialogTitle,+} from "@/components/ui/dialog";+import { useAuth } from "@/contexts/auth-context";+import type { CatalogCredentialField } from "../_helpers/catalog-models";+import {+ getProviderChrome,+ type ModelProvider,+} from "../_helpers/model-helpers";+import ModelProviderDialogFooter from "./model-provider-dialog-footer";+import {+ ProviderSettingsForm,+ type ProviderSettingsFormData,+} from "./provider-settings-form";++const EMPTY_FIELDS: CatalogCredentialField[] = [];++/**+ * Credential dialog for any provider without a bespoke one.+ *+ * Fields come from the catalogue's per-provider spec and are saved through the+ * generic `provider_credentials` payload, so adding a provider row to+ * `config/model_providers.yaml` is enough to make it configurable here.+ *+ * There is no live key check on save — OpenRAG has no generic+ * "list this provider's models" endpoint to probe with. The credentials are+ * validated the first time a model from this provider is selected in Agent or+ * Ingestion settings, which reports the provider's own error.+ */+const ProviderSettingsDialog = ({+ provider,+ displayName,+ open,+ setOpen,+}: {+ provider: ModelProvider;+ displayName?: string;+ open: boolean;+ setOpen: (open: boolean) => void;+}) => {+ const { isAuthenticated, isNoAuthMode } = useAuth();+ const [showRemoveConfirm, setShowRemoveConfirm] = useState(false);+ const [affectedModels, setAffectedModels] = useState<+ AffectedEmbeddingModel[] | undefined+ >(undefined);+ const router = useRouter();++ const { data: settings = {} } = useGetSettingsQuery({+ enabled: isAuthenticated || isNoAuthMode,+ });+ const { data: catalog } = useGetModelCatalogQuery({+ enabled: isAuthenticated || isNoAuthMode,+ });++ const chrome = getProviderChrome(provider, displayName);+ const catalogEntry = catalog?.providers?.find(+ (entry) => entry.key === provider,+ );+ const fields = catalogEntry?.credential_fields ?? EMPTY_FIELDS;++ const saved = settings.providers?.custom?.[provider];+ const isConfigured = saved?.configured === true;+ const savedSecretFields = saved?.secret_fields ?? [];++ // Removing the last configured provider would leave the agent with nothing+ // to call, so mirror the bespoke dialogs and require another one first.+ const canRemove =+ isConfigured &&+ Object.entries(settings.providers?.custom ?? {}).some(+ ([key, value]) => key !== provider && value?.configured === true,+ );++ const methods = useForm<ProviderSettingsFormData>({+ mode: "onSubmit",+ defaultValues: { credentials: {} },+ });++ // Seed on open, and again if the catalogue or the saved settings land after+ // it. Both are react-query results, so their identity is stable while the+ // dialog is open and this cannot stomp on what the user is typing.+ useEffect(() => {+ if (!open) {+ return;+ }+ methods.reset({+ credentials: Object.fromEntries(+ fields.map((field) => [+ field.key,+ saved?.credential_values?.[field.key] ?? "",+ ]),+ ),+ });+ }, [open, fields, saved, methods]);++ const { handleSubmit } = methods;++ const settingsMutation = useUpdateSettingsMutation({+ onSuccess: () => {+ toast.message(`${chrome.name} successfully configured`, {+ description: "You can now select its models in Settings.",+ duration: Infinity,+ closeButton: true,+ action: {+ label: "Settings",+ onClick: () => {+ router.push("/settings/agent?focusLlmModel=true");+ },+ },+ });+ setOpen(false);+ },+ });++ const removeMutation = useUpdateSettingsMutation({+ onSuccess: () => {+ toast.success(`${chrome.name} configuration removed`);+ setShowRemoveConfirm(false);+ setAffectedModels(undefined);+ setOpen(false);+ },+ onError: (err) => {+ if (isEmbeddingProviderInUseError(err)) {+ setAffectedModels(err.affectedModels);+ }+ },+ });++ const onSubmit = (data: ProviderSettingsFormData) => {+ // Blank means "leave the stored value alone": the backend ignores empty+ // values, and secrets are never echoed back for us to resubmit.+ const credentials: Record<string, string> = {};+ for (const [key, value] of Object.entries(data.credentials ?? {})) {+ const trimmed = (value ?? "").trim();+ if (trimmed !== "") {+ credentials[key] = trimmed;+ }+ }++ if (Object.keys(credentials).length === 0) {+ methods.setError("root", { message: "Enter at least one credential" });+ return;+ }++ settingsMutation.mutate({+ provider_credentials: { [provider]: credentials },+ });+ };++ const Logo = chrome.logo;++ return (+ <Dialog+ open={open}+ onOpenChange={(o) => {+ setShowRemoveConfirm(false);+ setAffectedModels(undefined);+ setOpen(o);+ }}+ >+ <DialogContent autoFocus={false} className="max-w-2xl overflow-hidden">+ <FormProvider {...methods}>+ <form+ onSubmit={handleSubmit(onSubmit)}+ className="grid min-w-0 gap-4"+ >+ <DialogHeader className="mb-2">+ <DialogTitle className="flex items-center gap-3">+ <div className="w-8 h-8 rounded flex items-center justify-center bg-white border">+ <Logo className="w-4 h-4 text-black" />+ </div>+ {chrome.name} Setup+ </DialogTitle>+ </DialogHeader>++ <ProviderSettingsForm+ providerName={chrome.name}+ fields={fields}+ savedSecretFields={savedSecretFields}+ saveError={methods.formState.errors.root?.message}+ />++ <LazyMotion features={domAnimation}>+ <AnimatePresence mode="wait">+ {settingsMutation.isError && (+ <m.div+ key="error"+ initial={{ opacity: 0, y: 10 }}+ animate={{ opacity: 1, y: 0 }}+ exit={{ opacity: 0, y: -10 }}+ >+ <p className="rounded-lg border border-destructive p-4 min-w-0 [overflow-wrap:anywhere]">+ {settingsMutation.error?.message}+ </p>+ </m.div>+ )}+ {removeMutation.isError &&+ !isEmbeddingProviderInUseError(removeMutation.error) && (+ <m.div+ key="remove-error"+ initial={{ opacity: 0, y: 10 }}+ animate={{ opacity: 1, y: 0 }}+ exit={{ opacity: 0, y: -10 }}+ >+ <p className="rounded-lg border border-destructive p-4 min-w-0 [overflow-wrap:anywhere]">+ {removeMutation.error?.message}+ </p>+ </m.div>+ )}+ </AnimatePresence>+ </LazyMotion>++ <ModelProviderDialogFooter+ showRemoveConfirm={showRemoveConfirm}+ onCancelRemove={() => {+ setShowRemoveConfirm(false);+ setAffectedModels(undefined);+ }}+ onConfirmRemove={() =>+ removeMutation.mutate({+ remove_provider_config: provider,+ force_remove: !!affectedModels,+ })+ }+ isRemovePending={removeMutation.isPending}+ isConfigured={isConfigured}+ canRemove={canRemove}+ providerKey={provider}+ removeDisabledTooltip={`Configure another model provider before removing ${chrome.name}`}+ onRequestRemove={() => setShowRemoveConfirm(true)}+ onCancel={() => setOpen(false)}+ isSavePending={settingsMutation.isPending}+ isValidating={false}+ affectedModels={affectedModels}+ />+ </form>+ </FormProvider>+ </DialogContent>+ </Dialog>+ );+};++export default ProviderSettingsDialog;diff --git a/frontend/app/settings/_components/provider-settings-form.tsx b/frontend/app/settings/_components/provider-settings-form.tsxnew file mode 100644index 000000000..03091140b--- /dev/null+++ b/frontend/app/settings/_components/provider-settings-form.tsx@@ -0,0 +1,113 @@+import { useFormContext } from "react-hook-form";+import type { CatalogCredentialField } from "@/app/settings/_helpers/catalog-models";+import { LabelWrapper } from "@/components/label-wrapper";+import { Input } from "@/components/ui/input";+import { Textarea } from "@/components/ui/textarea";++export interface ProviderSettingsFormData {+ credentials: Record<string, string>;+}++/**+ * Credential form for a provider with no bespoke dialog, rendered from the+ * field spec `/api/models/catalog` publishes for it. A provider added to+ * `config/model_providers.yaml` is configurable through this without any+ * further frontend work.+ */+export function ProviderSettingsForm({+ providerName,+ fields,+ savedSecretFields,+ saveError,+}: {+ providerName: string;+ fields: CatalogCredentialField[];+ savedSecretFields: string[];+ saveError?: string | null;+}) {+ const {+ register,+ formState: { errors },+ } = useFormContext<ProviderSettingsFormData>();++ const saved = new Set(savedSecretFields);++ return (+ <div className="min-w-0 space-y-4">+ {fields.map((field) => {+ const error = errors.credentials?.[field.key]?.message;+ const isSecret =+ field.field_type === "password" || field.field_type === "textarea";+ const hasSaved = isSecret && saved.has(field.key);+ // A stored secret is never sent back to the browser, so an empty box+ // means "keep what is saved" rather than "no value" — required only+ // bites when nothing is stored yet.+ const required = field.required && !hasSaved;++ return (+ <div key={field.key} className="min-w-0 space-y-2">+ <LabelWrapper+ label={field.label}+ helperText={field.tooltip ?? undefined}+ required={required}+ id={`provider-field-${field.key}`}+ >+ {field.field_type === "textarea" ? (+ <Textarea+ {...register(`credentials.${field.key}`, {+ required: required ? `${field.label} is required` : false,+ })}+ className={error ? "!border-destructive" : ""}+ id={`provider-field-${field.key}`}+ placeholder={+ hasSaved ? "•••••••••" : (field.placeholder ?? undefined)+ }+ />+ ) : (+ <Input+ {...register(`credentials.${field.key}`, {+ required: required ? `${field.label} is required` : false,+ })}+ className={error ? "!border-destructive" : ""}+ id={`provider-field-${field.key}`}+ type={field.field_type === "password" ? "password" : "text"}+ autoComplete={+ field.field_type === "password" ? "new-password" : "off"+ }+ placeholder={+ hasSaved ? "•••••••••" : (field.placeholder ?? undefined)+ }+ />+ )}+ </LabelWrapper>+ {hasSaved && (+ <p className="text-sm text-muted-foreground">+ A value is already saved. Leave this blank to keep it.+ </p>+ )}+ {error && (+ <p+ data-testid="provider-connection-error"+ className="text-sm text-destructive min-w-0 [overflow-wrap:anywhere]"+ >+ {error}+ </p>+ )}+ </div>+ );+ })}+ {saveError && (+ <p+ data-testid="provider-connection-error"+ className="text-sm text-destructive min-w-0 [overflow-wrap:anywhere]"+ >+ {saveError}+ </p>+ )}+ <p className="text-sm text-muted-foreground">+ Pick the {providerName} models to use from the Agent and Ingestion+ settings after saving these credentials.+ </p>+ </div>+ );+}diff --git a/frontend/app/settings/_helpers/catalog-models.test.ts b/frontend/app/settings/_helpers/catalog-models.test.tsindex 4a9906f25..a698deca2 100644--- a/frontend/app/settings/_helpers/catalog-models.test.ts+++ b/frontend/app/settings/_helpers/catalog-models.test.ts@@ -3,6 +3,7 @@ import { describe, it } from "node:test";import {findGroupedSelection,groupedCatalogOptions,+ liveModelOption,mergeLiveCatalogOptions,onboardingCatalogConfigured,onboardingCredentialFields,@@ -280,3 +281,70 @@ describe("providerCredentialsSatisfied", () => {);});});++describe("newest models first", () => {+ const provider = (models: Array<{ model: string }>) => ({+ providers: [{ key: "p", name: "P", models, embedding_models: [] }],+ });+ const order = (names: string[]) =>+ groupedCatalogOptions(+ provider(+ names.map((model) => ({ model, capabilities: ["function_calling"] })),+ ),+ undefined,+ "language",+ )[0].options.map((option) => option.value);++ it("ranks by version, newest first", () => {+ assert.deepEqual(order(["gpt-4o", "gpt-3.5-turbo", "gpt-5", "gpt-4.1"]), [+ "gpt-5",+ "gpt-4.1",+ "gpt-4o",+ "gpt-3.5-turbo",+ ]);+ });++ it("puts a bare alias above its own dated snapshot", () => {+ // `gpt-5` and `gpt-5-2025-08-07` are the same model; the alias is the one+ // to offer, so a missing stamp has to rank as newer, not older.+ assert.deepEqual(order(["gpt-5-2025-08-07", "gpt-5"]), [+ "gpt-5",+ "gpt-5-2025-08-07",+ ]);+ });++ it("orders snapshots of one version by their stamp", () => {+ assert.deepEqual(+ order(["claude-opus-4-5-20251101", "claude-opus-4-5-20260401"]),+ ["claude-opus-4-5-20260401", "claude-opus-4-5-20251101"],+ );+ });++ it("falls back to alphabetical when nothing is versioned", () => {+ assert.deepEqual(order(["mistral-large", "granite-instruct"]), [+ "granite-instruct",+ "mistral-large",+ ]);+ });+});++describe("mergeLiveCatalogOptions creates a missing group", () => {+ it("adds watsonx embeddings the bundled table does not carry", () => {+ // LiteLLM lists zero watsonx embedding models, so groupedCatalogOptions+ // drops the group entirely — the live rows are the only ones there are.+ const merged = mergeLiveCatalogOptions(+ [],+ "watsonx",+ [liveModelOption("ibm/slate-125m-english-rtrvr", "watsonx")],+ "IBM watsonx.ai",+ );++ assert.equal(merged.length, 1);+ assert.equal(merged[0].key, "watsonx");+ assert.equal(merged[0].group, "IBM watsonx.ai");+ assert.deepEqual(+ merged[0].options.map((option) => option.value),+ ["ibm/slate-125m-english-rtrvr"],+ );+ });+});diff --git a/frontend/app/settings/_helpers/catalog-models.ts b/frontend/app/settings/_helpers/catalog-models.tsindex 2984a2058..0675f2d94 100644--- a/frontend/app/settings/_helpers/catalog-models.ts+++ b/frontend/app/settings/_helpers/catalog-models.ts@@ -92,12 +92,46 @@ const PREFERRED_EMBEDDING_MODELS = ["text-embedding-3-large",];+/** A YYYY-MM-DD or YYYYMMDD snapshot stamp inside a model id. */+const SNAPSHOT_DATE = /(20\d{2})-?(\d{2})-?(\d{2})(?!\d)/;++/**+ * How recent a model id looks, most-recent-first. Nothing in LiteLLM's data+ * says when a model shipped — `deprecation_date` is the only date it carries,+ * and it is not a stand-in: Anthropic retires on a rolling year so it tracks+ * release almost exactly, but OpenAI sunsets a decade of models on one shared+ * date, which would rank gpt-3.5-turbo above gpt-5. So recency is read off the+ * id itself.+ *+ * Version before snapshot, deliberately. `gpt-5` and `gpt-5-2025-08-07` are+ * the same model, and the bare alias is the one to offer — so an id with no+ * stamp sorts *above* its dated snapshots rather than below them.+ *+ * Providers that number by size rather than version (watsonx's `jais-13b`,+ * `granite-3-8b`) get an ordering that means nothing in particular. It is+ * stable and alphabetical within a version, which is all it needs to be.+ */+function recencyRank(name: string): Array<number> {+ const stamp = SNAPSHOT_DATE.exec(name);+ // Strip the stamp before reading version digits, or 2025 becomes a version.+ const withoutStamp = stamp ? name.replace(stamp[0], "") : name;+ const version = (withoutStamp.match(/\d+(?:\.\d+)?/g) ?? [])+ .slice(0, 3)+ .map(Number);+ while (version.length < 3) version.push(-1);++ const stampRank = stamp+ ? -Number(`${stamp[1]}${stamp[2]}${stamp[3]}`)+ : -Number.MAX_SAFE_INTEGER;++ return [-version[0], -version[1], -version[2], stampRank];+}+function optionRank(option: CatalogSelectOption,kind: CatalogModelKind,): Array<number | string> {const name = option.value;- const lower = name.toLowerCase();const caps = option.model?.capabilities ?? [];const preferred =kind === "embedding"@@ -105,10 +139,17 @@ function optionRank(: PREFERRED_LANGUAGE_MODELS;const preferredIndex = preferred.indexOf(name);return [- lower.startsWith("ft:") ? 1 : 0,+ // The catalogue no longer carries LiteLLM's `ft:` pricing templates, but a+ // live `/models/openai` fetch returns the operator's real fine-tunes. Those+ // are callable and stay listed — just never above a base model.+ name.toLowerCase().startsWith("ft:") ? 1 : 0,caps.length === 0 ? 1 : 0,kind === "language" && !caps.includes("function_calling") ? 1 : 0,kind === "vision" && !caps.includes("vision") ? 1 : 0,+ ...recencyRank(name),+ // Only reached by ids that look equally recent: the embedding lists are+ // undated and unversioned, so this is what still puts a validated default+ // on top there.preferredIndex === -1 ? preferred.length : preferredIndex,name,];@@ -223,15 +264,32 @@ export function onboardingCatalogConfigured(return undefined;}-/** Append live /models/{provider} rows that the catalogue does not already list. */+/**+ * Append live `/models/{provider}` rows that the catalogue does not list.+ *+ * Some providers' inventory is only knowable from the running server: watsonx+ * publishes *no* embedding models in LiteLLM's bundled table at all, and+ * Ollama serves whatever the operator pulled. Without this the watsonx group+ * is empty for embeddings, and `groupedCatalogOptions` then drops the group+ * entirely — which is why watsonx was missing from the embedding picker+ * rather than merely short of options. So the merge creates the group when+ * the catalogue produced none.+ */export function mergeLiveCatalogOptions(groups: CatalogOptionGroup[],provider: string | undefined,live: CatalogSelectOption[],+ displayName?: string,): CatalogOptionGroup[] {if (!provider || live.length === 0) {return groups;}+ if (!groups.some((group) => group.key === provider)) {+ return [+ ...groups,+ { key: provider, group: displayName || provider, options: live },+ ];+ }return groups.map((group) => {if (group.key !== provider) {return group;@@ -245,6 +303,21 @@ export function mergeLiveCatalogOptions(});}+/**+ * Providers whose model list comes from the running server, not from+ * LiteLLM's bundled table. watsonx publishes no embedding models in that+ * table at all, and Ollama serves whatever the operator pulled locally.+ */+export const LIVE_INVENTORY_PROVIDERS = new Set(["watsonx", "ollama"]);++/** A live `/models/{provider}` row as a picker option. */+export function liveModelOption(+ model: string,+ provider: string,+): CatalogSelectOption {+ return { value: model, label: model, provider };+}+/*** Credential controls for onboarding come directly from LiteLLM's provider* schema. The backend persists and encrypts arbitrary field keys.diff --git a/frontend/app/settings/_helpers/model-helpers.test.ts b/frontend/app/settings/_helpers/model-helpers.test.tsnew file mode 100644index 000000000..3c959e287--- /dev/null+++ b/frontend/app/settings/_helpers/model-helpers.test.ts@@ -0,0 +1,81 @@+/**+ * Provider ordering and chrome.+ *+ * Which providers exist is the backend's call (config/model_providers.yaml+ * filtered by OPENRAG_RUN_MODE); the frontend only decides what order to show+ * them in and what to call them. These pin that a provider the backend adds+ * still renders, and that a hidden one can never be re-introduced by the+ * preference lists.+ */++import assert from "node:assert/strict";+import { describe, it } from "node:test";+import {+ EMBEDDING_PROVIDER_ORDER,+ getProviderChrome,+ LLM_PROVIDER_ORDER,+ orderProviders,+} from "./model-helpers";++describe("orderProviders", () => {+ it("ranks preferred providers first and keeps the rest in API order", () => {+ assert.deepEqual(+ orderProviders(+ ["openai", "watsonx", "azure_ai", "anthropic"],+ ["anthropic", "openai"],+ ),+ ["anthropic", "openai", "watsonx", "azure_ai"],+ );+ });++ it("never adds a provider the backend did not offer", () => {+ // SaaS hides Ollama, so no preference list may put it back.+ assert.deepEqual(+ orderProviders(["openai", "watsonx"], LLM_PROVIDER_ORDER),+ ["openai", "watsonx"],+ );+ assert.deepEqual(orderProviders(["openai"], EMBEDDING_PROVIDER_ORDER), [+ "openai",+ ]);+ });++ it("keeps a provider that no preference list mentions", () => {+ assert.deepEqual(orderProviders(["azure_ai"], LLM_PROVIDER_ORDER), [+ "azure_ai",+ ]);+ });+});++describe("getProviderChrome", () => {+ it("names an unknown provider by its API display name", () => {+ assert.equal(getProviderChrome("groq", "Groq Cloud").name, "Groq Cloud");+ });++ it("falls back to the provider key when the API sent no name", () => {+ assert.equal(getProviderChrome("groq").name, "groq");+ });++ it("lets the API display name override a built-in label", () => {+ assert.equal(getProviderChrome("openai").name, "OpenAI");+ assert.equal(+ getProviderChrome("openai", "House Gateway").name,+ "House Gateway",+ );+ });++ it("gives each Azure service its own Microsoft mark", () => {+ // Foundry and Azure OpenAI are drawn differently by Microsoft, and neither+ // is the generic Azure logo. `azure` had no chrome row at all, so its card+ // fell through to the placeholder while the model rows beside it were+ // already branded.+ const foundry = getProviderChrome("azure_ai");+ const openai = getProviderChrome("azure");+ const placeholder = getProviderChrome("groq").logo;++ assert.equal(foundry.name, "Azure AI Foundry");+ assert.equal(openai.name, "Azure OpenAI");+ assert.notEqual(foundry.logo, openai.logo);+ assert.notEqual(foundry.logo, placeholder);+ assert.notEqual(openai.logo, placeholder);+ });+});diff --git a/frontend/app/settings/_helpers/model-helpers.tsx b/frontend/app/settings/_helpers/model-helpers.tsxindex cd8d4f3ef..96ffccb71 100644--- a/frontend/app/settings/_helpers/model-helpers.tsx+++ b/frontend/app/settings/_helpers/model-helpers.tsx@@ -1,24 +1,37 @@+import AiFoundryLogo from "@/components/icons/ai-foundry-logo";import AnthropicLogo from "@/components/icons/anthropic-logo";+import AzureOpenAILogo from "@/components/icons/azure-openai-logo";+import GenericProviderLogo from "@/components/icons/generic-provider-logo";import IBMLogo from "@/components/icons/ibm-logo";import OllamaLogo from "@/components/icons/ollama-logo";import OpenAILogo from "@/components/icons/openai-logo";-export type ModelProvider =- | "openai"- | "anthropic"- | "ollama"- | "watsonx"- | "local";+/**+ * A provider key as the backend names it.+ *+ * Which providers exist — and which of them this deployment shows — comes from+ * `GET /api/models/providers`, driven by `config/model_providers.yaml` and+ * OPENRAG_RUN_MODE. So this is a plain string, not a closed union: a provider+ * added to that file must render without a frontend change. The keys below are+ * only the ones OpenRAG has bespoke chrome (logo, colours, forms) for.+ */+export type ModelProvider = string;-// Full ordered list of providers for settings / cards-export const ALL_PROVIDERS: ModelProvider[] = [+export const KNOWN_PROVIDERS = ["openai",+ "anthropic","ollama","watsonx",- "anthropic",-];+ "azure_ai",+ "azure",+ "local",+] as const;-// Preferred auto-select order for the LLM onboarding step+export type KnownModelProvider = (typeof KNOWN_PROVIDERS)[number];++// Preferred auto-select order for the LLM onboarding step. Only a preference:+// providers this run mode hides are dropped, and anything the API returns that+// is not listed here is appended in API order.export const LLM_PROVIDER_ORDER: ModelProvider[] = ["anthropic","openai",@@ -33,14 +46,107 @@ export const EMBEDDING_PROVIDER_ORDER: ModelProvider[] = ["ollama",];-// Providers unavailable in cloud (IBM) deployments-export const CLOUD_EXCLUDED_PROVIDERS: ModelProvider[] = ["ollama"];+/**+ * `preferred` first (skipping anything not in `available`), then whatever else+ * `available` holds, in the order the backend returned it.+ */+export function orderProviders(+ available: ModelProvider[],+ preferred: ModelProvider[],+): ModelProvider[] {+ const offered = new Set(available);+ const ranked = preferred.filter((provider) => offered.has(provider));+ const seen = new Set(ranked);+ return [...ranked, ...available.filter((provider) => !seen.has(provider))];+}export interface ModelOption {value: string;label: string;}+export interface ProviderChrome {+ /** Display name; the API's `display_name` wins over the built-in label. */+ name: string;+ logo: (props: React.SVGProps<SVGSVGElement>) => React.ReactNode;+ logoColor: string;+ logoBgColor: string;+ /** Onboarding tabs invert some marks; defaults to the card colours. */+ tabLogoColor?: string;+ tabLogoBgColor?: string;+}++/** Logo and colours for the providers OpenRAG ships chrome for. */+const PROVIDER_CHROME: Record<string, ProviderChrome> = {+ openai: {+ name: "OpenAI",+ logo: OpenAILogo,+ logoColor: "text-black",+ logoBgColor: "bg-white",+ },+ anthropic: {+ name: "Anthropic",+ logo: AnthropicLogo,+ logoColor: "text-[#D97757]",+ logoBgColor: "bg-white",+ tabLogoColor: "text-black",+ tabLogoBgColor: "bg-[#D97757]",+ },+ ollama: {+ name: "Ollama",+ logo: OllamaLogo,+ logoColor: "text-black",+ logoBgColor: "bg-white",+ },+ watsonx: {+ name: "IBM watsonx.ai",+ logo: IBMLogo,+ logoColor: "text-white",+ logoBgColor: "bg-[#1063FE]",+ },+ // Microsoft draws the two Azure model services differently, and neither is+ // the generic Azure logo. Both marks paint their own gradients, so logoColor+ // does nothing for them and the tile stays white.+ azure_ai: {+ name: "Azure AI Foundry",+ logo: AiFoundryLogo,+ logoColor: "text-black",+ logoBgColor: "bg-white",+ },+ azure: {+ name: "Azure OpenAI",+ logo: AzureOpenAILogo,+ logoColor: "text-black",+ logoBgColor: "bg-white",+ },+ local: {+ name: "Local",+ logo: GenericProviderLogo,+ logoColor: "text-muted-foreground",+ logoBgColor: "bg-white",+ },+};++/**+ * Chrome for `provider`. A provider the config file adds but the frontend has+ * no artwork for still renders, under the display name the API gave it.+ */+export function getProviderChrome(+ provider: ModelProvider,+ displayName?: string,+): ProviderChrome {+ const known = PROVIDER_CHROME[provider];+ if (known) {+ return displayName ? { ...known, name: displayName } : known;+ }+ return {+ name: displayName || provider,+ logo: GenericProviderLogo,+ logoColor: "text-black",+ logoBgColor: "bg-white",+ };+}+// Helper function to get model logo based on provider or model nameexport function getModelLogo(modelValue: string, provider?: string) {// First check by provider@@ -52,6 +158,10 @@ export function getModelLogo(modelValue: string, provider?: string) {return <OllamaLogo className="w-4 h-4" />;} else if (provider === "watsonx") {return <IBMLogo className="w-4 h-4" />;+ } else if (provider === "azure") {+ return <AzureOpenAILogo className="w-4 h-4" />;+ } else if (provider === "azure_ai") {+ return <AiFoundryLogo className="w-4 h-4" />;} else if (provider === "local") {return (<svgdiff --git a/frontend/components/icons/ai-foundry-logo.tsx b/frontend/components/icons/ai-foundry-logo.tsxnew file mode 100644index 000000000..2d627a682--- /dev/null+++ b/frontend/components/icons/ai-foundry-logo.tsx@@ -0,0 +1,102 @@+/**+ * Azure AI Foundry, as Microsoft draws it (az-icons.com/icon/ai-foundry).+ *+ * Distinct from `azure-openai-logo`: Microsoft draws the two Azure model+ * services differently. Gradient ids are fixed rather than generated: every+ * instance defines the+ * same stops, so when several render on one page and the browser resolves+ * `url(#...)` to the first match, they all paint identically.+ */+export default function AiFoundryLogo(props: React.SVGProps<SVGSVGElement>) {+ return (+ <svg+ xmlns="http://www.w3.org/2000/svg"+ width="16"+ height="16"+ viewBox="0 0 18 18"+ fill="none"+ {...props}+ >+ <title>Azure AI Foundry Logo</title>+ <defs>+ <linearGradient+ id="ai-foundry-flame-base"+ x1="10.91"+ x2="13.62"+ y1="13.66"+ y2="13.66"+ gradientTransform="matrix(1 0 0 -1 0 20)"+ gradientUnits="userSpaceOnUse"+ >+ <stop offset="0" stopColor="#302ec9" />+ <stop offset=".12" stopColor="#151698" />+ <stop offset=".25" stopColor="#0a0b7e" />+ <stop offset=".43" stopColor="#090c7f" />+ <stop offset=".51" stopColor="#141698" />+ <stop offset=".61" stopColor="#282ac4" />+ <stop offset=".77" stopColor="#8e8ff4" />+ <stop offset=".85" stopColor="#a7a6f7" />+ <stop offset=".91" stopColor="#cacafb" />+ <stop offset="1" stopColor="#fff" />+ </linearGradient>+ <linearGradient+ id="ai-foundry-flame-overlay"+ x1="12.26"+ x2="12.26"+ y1="8.32"+ y2="18.23"+ gradientTransform="matrix(1 0 0 -1 0 20)"+ gradientUnits="userSpaceOnUse"+ >+ <stop offset=".02" stopColor="#201ba6" />+ <stop offset=".64" stopColor="#2d29c7" />+ <stop offset="1" stopColor="#201ba6" />+ </linearGradient>+ <linearGradient+ id="ai-foundry-crucible"+ x1="14.19"+ x2="14.19"+ y1="8.27"+ y2="14.47"+ gradientTransform="matrix(1 0 0 -1 0 20)"+ gradientUnits="userSpaceOnUse"+ >+ <stop offset="0" stopColor="#3530c3" />+ <stop offset=".5" stopColor="#6d71d1" />+ <stop offset=".97" stopColor="#c0bff5" />+ <stop offset="1" stopColor="#e3e4ff" />+ </linearGradient>+ <linearGradient+ id="ai-foundry-body"+ x1="6.51"+ x2="6.51"+ y1="3"+ y2="19.15"+ gradientTransform="matrix(1 0 0 -1 0 20)"+ gradientUnits="userSpaceOnUse"+ >+ <stop offset="0" stopColor="#302ec9" />+ <stop offset=".45" stopColor="#302ec9" />+ <stop offset=".95" stopColor="#cacafb" />+ <stop offset="1" stopColor="#7f7eaf" />+ </linearGradient>+ </defs>+ <path+ fill="url(#ai-foundry-flame-base)"+ d="M11.96 11.68s.07-.1.07-.24V7.57c0-.85.66-1.76 1.59-2.03-.06-.19-.64-2.11-.81-2.65-.17-.56-.67-1.9-1.29-1.9h-.01s-.09.05-.13.13c-.32.58-.46 3.02-.46 4.84v2.26c.27.97.91 3.22.96 3.35 0 0 .05.1.09.1Z"+ />+ <path+ fill="url(#ai-foundry-flame-overlay)"+ d="M11.96 11.68s.07-.1.07-.24V7.57c0-.85.66-1.76 1.59-2.03-.06-.19-.64-2.11-.81-2.65-.17-.56-.67-1.9-1.29-1.9h-.01s-.09.05-.13.13c-.32.58-.46 3.02-.46 4.84v2.26c.27.97.91 3.22.96 3.35 0 0 .05.1.09.1Z"+ />+ <path+ fill="url(#ai-foundry-crucible)"+ d="M16.01 5.46H14.2c-1.21 0-2.17 1.09-2.17 2.11v3.87c0 .14-.04.24-.07.24s-.09-.1-.09-.1.07.21.17.21h1.98c.62 0 2.48-.61 2.48-2.52V6.02c0-.41-.24-.55-.49-.55Z"+ />+ <path+ fill="url(#ai-foundry-body)"+ d="M2.34 17h5.38c2.35 0 3.2-2.03 3.2-3.81V5.96c0-2.08.19-4.97.61-4.97H8.3c-.77 0-1.43.82-2.18 2.48S1.76 15.34 1.6 15.83c-.26.79-.02 1.16.74 1.16Z"+ />+ </svg>+ );+}diff --git a/frontend/components/icons/azure-openai-logo.tsx b/frontend/components/icons/azure-openai-logo.tsxnew file mode 100644index 000000000..8b8762f4b--- /dev/null+++ b/frontend/components/icons/azure-openai-logo.tsx@@ -0,0 +1,40 @@+/**+ * Azure OpenAI Service, as Microsoft draws it (az-icons.com/icon/azure-openai).+ *+ * Distinct from `ai-foundry-logo`: Microsoft draws the two Azure model services+ * differently. The gradient id is fixed rather than generated: every instance+ * defines the same stops, so when a page renders several of these and the+ * browser resolves `url(#...)` to the first one, they all paint identically.+ */+export default function AzureOpenAILogo(props: React.SVGProps<SVGSVGElement>) {+ return (+ <svg+ xmlns="http://www.w3.org/2000/svg"+ width="16"+ height="16"+ viewBox="0 0 18 18"+ fill="none"+ {...props}+ >+ <title>Azure OpenAI Logo</title>+ <defs>+ <radialGradient+ id="azure-openai-mark"+ cx="-67.981"+ cy="793.199"+ r=".45"+ gradientTransform="translate(-17939.03 20368.029) rotate(45) scale(25.091 -34.149)"+ gradientUnits="userSpaceOnUse"+ >+ <stop offset="0" stopColor="#83b9f9" />+ <stop offset="1" stopColor="#0078d4" />+ </radialGradient>+ </defs>+ <path+ d="m0,2.7v12.6c0,1.491,1.209,2.7,2.7,2.7h12.6c1.491,0,2.7-1.209,2.7-2.7V2.7c0-1.491-1.209-2.7-2.7-2.7H2.7C1.209,0,0,1.209,0,2.7ZM10.8,0v3.6c0,3.976,3.224,7.2,7.2,7.2h-3.6c-3.976,0-7.199,3.222-7.2,7.198v-3.598c0-3.976-3.224-7.2-7.2-7.2h3.6c3.976,0,7.2-3.224,7.2-7.2Z"+ fill="url(#azure-openai-mark)"+ strokeWidth="0"+ />+ </svg>+ );+}diff --git a/frontend/components/icons/generic-provider-logo.tsx b/frontend/components/icons/generic-provider-logo.tsxnew file mode 100644index 000000000..ed447e1c9--- /dev/null+++ b/frontend/components/icons/generic-provider-logo.tsx@@ -0,0 +1,21 @@+export default function GenericProviderLogo(+ props: React.SVGProps<SVGSVGElement>,+) {+ return (+ <svg+ xmlns="http://www.w3.org/2000/svg"+ viewBox="0 0 24 24"+ fill="none"+ stroke="currentColor"+ strokeWidth="2"+ strokeLinecap="round"+ strokeLinejoin="round"+ {...props}+ >+ <title>Model provider</title>+ <path d="M21 16V8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16z" />+ <polyline points="3.27 6.96 12 12.01 20.73 6.96" />+ <line x1="12" y1="22.08" x2="12" y2="12" />+ </svg>+ );+}diff --git a/frontend/components/provider-health-banner.tsx b/frontend/components/provider-health-banner.tsxindex 749252521..655bc4e7e 100644--- a/frontend/components/provider-health-banner.tsx+++ b/frontend/components/provider-health-banner.tsx@@ -3,7 +3,7 @@import { AlertTriangle } from "lucide-react";import { useRouter } from "next/navigation";import { useProviderHealthQuery } from "@/app/api/queries/useProviderHealthQuery";-import type { ModelProvider } from "@/app/settings/_helpers/model-helpers";+import { getProviderChrome } from "@/app/settings/_helpers/model-helpers";import { Banner, BannerIcon, BannerTitle } from "@/components/ui/banner";import { useChat } from "@/contexts/chat-context";import { cn } from "@/lib/utils";@@ -48,14 +48,6 @@ export function useProviderHealth() {};}-const providerTitleMap: Record<ModelProvider, string> = {- openai: "OpenAI",- anthropic: "Anthropic",- ollama: "Ollama",- watsonx: "IBM watsonx.ai",- local: "Local",-};-export function ProviderHealthBanner({ className }: ProviderHealthBannerProps) {const { isLoading, isHealthy, isUnhealthy, health } = useProviderHealth();const router = useRouter();@@ -100,8 +92,10 @@ export function ProviderHealthBanner({ className }: ProviderHealthBannerProps) {errorProvider = llmProvider;}+ // One label source for every provider, including ones added through+ // config/model_providers.yaml that have no built-in chrome.const providerTitle = errorProvider- ? providerTitleMap[errorProvider as ModelProvider] || errorProvider+ ? getProviderChrome(errorProvider).name: "Provider";const settingsUrl = errorProviderdiff --git a/frontend/tests/pages/Settings.ts b/frontend/tests/pages/Settings.tsindex 24614b651..e4be8ec18 100644--- a/frontend/tests/pages/Settings.ts+++ b/frontend/tests/pages/Settings.ts@@ -52,16 +52,34 @@ export class Settings {.getByTestId("provider-connection-error").first();+ /**+ * Get locator for a provider's card.+ *+ * `hasText` matches substrings, so filtering on "OpenAI" also selected the+ * "Azure OpenAI" tile once both providers shipped, and any button lookup+ * inside it blew up under strict mode. Anchor on the card heading, which+ * carries the provider name exactly.+ * @param providerName - Name of the model provider+ * @returns Locator for the provider card+ */+ private getProviderCard(providerName: string) {+ return this.page.locator("div.rounded-xl, div.border-border.group").filter({+ has: this.page.getByRole("heading", {+ name: providerName,+ exact: true,+ }),+ });+ }+/*** Get locator for configure button by provider name* @param providerName - Name of the model provider* @returns Locator for the configure button*/private getConfigureButton(providerName: string) {- return this.page- .locator("div.rounded-xl, div.border-border.group")- .filter({ hasText: providerName })- .getByRole("button", { name: "Configure" });+ return this.getProviderCard(providerName).getByRole("button", {+ name: "Configure",+ });}/**@@ -100,10 +118,9 @@ export class Settings {* @returns Locator for the edit setup button*/private getEditSetupButton(providerName: string) {- return this.page- .locator("div.rounded-xl, div.border-border.group")- .filter({ hasText: providerName })- .getByRole("button", { name: "Edit Setup" });+ return this.getProviderCard(providerName).getByRole("button", {+ name: "Edit Setup",+ });}/**@@ -143,9 +160,7 @@ export class Settings {* @returns Locator for the search input*/private getSearchModelInput() {- let search = this.page.locator(- 'input[placeholder="Search model..."]:focus',- );+ let search = this.page.locator('[data-testid="model-search-input"]:focus');return search;}@@ -154,7 +169,7 @@ export class Settings {* @returns Locator for the search input*/private getSearchModelInputFallback() {- return this.page.locator('input[placeholder="Search model..."]').first();+ return this.page.locator('[data-testid="model-search-input"]').first();}/**diff --git a/src/api/models.py b/src/api/models.pyindex 308c8e17c..d06c4122e 100644--- a/src/api/models.py+++ b/src/api/models.py@@ -11,6 +11,7 @@ from api.provider_validation import ()from config.settings import get_openrag_configfrom dependencies import get_models_service, require_permission+from services.model_catalog import hide_excluded_live_modelsfrom session_manager import Userfrom utils.logging_config import get_logger@@ -97,7 +98,7 @@ async def get_openai_models()models = await models_service.get_openai_models(api_key=api_key)- return JSONResponse(models)+ return JSONResponse(hide_excluded_live_models("openai", models))except Exception as e:logger.error(f"Failed to get OpenAI models: {str(e)}")return _models_error_response(e)@@ -127,7 +128,7 @@ async def get_anthropic_models()models = await models_service.get_anthropic_models(api_key=api_key)- return JSONResponse(models)+ return JSONResponse(hide_excluded_live_models("anthropic", models))except Exception as e:logger.error(f"Failed to get Anthropic models: {str(e)}")return _models_error_response(e)@@ -154,7 +155,7 @@ async def get_ollama_models()models = await models_service.get_ollama_models(endpoint=endpoint)- return JSONResponse(models)+ return JSONResponse(hide_excluded_live_models("ollama", models))except Exception as e:logger.error(f"Failed to get Ollama models: {str(e)}")return _models_error_response(e)@@ -211,12 +212,35 @@ async def get_ibm_models(models = await models_service.get_ibm_models(endpoint=endpoint, api_key=api_key, project_id=project_id)- return JSONResponse(models)+ return JSONResponse(hide_excluded_live_models("watsonx", models))except Exception as e:logger.error(f"Failed to get IBM models: {str(e)}")return _models_error_response(e)+async def get_model_providers(+ user: User = Depends(require_permission("providers:read")),+):+ """Providers this run mode exposes. GET /models/providers++ The one list Settings cards, Onboarding tabs and the model pickers render.+ Unlike /models/catalog it does not need LiteLLM, so the UI can still lay out+ its provider surfaces when the catalogue is unavailable.+ """+ from config.model_providers import provider_visibility_payload++ # `no-store`, not a max-age: the list is derived from+ # `config/model_providers.yaml`, which an operator edits and then restarts+ # the backend for. A cached response outlives that restart in the browser,+ # so the console keeps drawing the old provider cards/tabs and the change+ # looks like it did nothing. The payload is a few hundred bytes and React+ # Query already holds it for the session, so the round trip costs nothing.+ return JSONResponse(+ provider_visibility_payload(),+ headers={"Cache-Control": "no-store"},+ )++async def get_model_catalog(user: User = Depends(require_permission("providers:read")),):@@ -227,8 +251,12 @@ async def get_model_catalog(catalog,)+ # `no-store` for the same reason as /models/providers: the catalogue is+ # filtered by that config file, so a cached copy survives the restart that+ # was supposed to apply the edit. LiteLLM's table is cached in-process, so+ # rebuilding this response is cheap.try:- return JSONResponse(catalog(), headers={"Cache-Control": "private, max-age=3600"})+ return JSONResponse(catalog(), headers={"Cache-Control": "no-store"})except CatalogUnavailableError as e:# Log the cause, but never echo exception text back to the caller# (CodeQL py/stack-trace-exposure).diff --git a/src/api/provider_health.py b/src/api/provider_health.pyindex 8e3b514e5..e5ccfb901 100644--- a/src/api/provider_health.py+++ b/src/api/provider_health.py@@ -9,6 +9,7 @@ from fastapi.responses import JSONResponsefrom api.provider_validation import sanitize_provider_error_content, validate_provider_setupfrom config.settings import get_openrag_configfrom dependencies import require_permission+from services import provider_error_logfrom services.model_catalog import is_known_providerfrom session_manager import Userfrom utils import provider_health_cache@@ -145,8 +146,17 @@ async def check_provider_health(embedding_project_id=embedding_project_id,embedding_credentials=embedding_credentials,)+ # A cached *healthy* verdict must not outlive a real failure. The+ # cache exists to coalesce identical probes, and a recorded failure+ # means traffic is failing right now regardless of what the last+ # probe concluded — so fall through and let the response below+ # report it.+ has_real_failure = bool(+ provider_error_log.latest_failure(provider, "chat")+ or provider_error_log.latest_failure(embedding_provider, "embedding")+ )cached_payload = provider_health_cache.get(health_cache_key)- if cached_payload is not None:+ if cached_payload is not None and not has_real_failure:logger.debug("Returning cached provider-health response")return JSONResponse(cached_payload, status_code=200)@@ -160,7 +170,7 @@ async def check_provider_health(break# Woke up after an in-flight validation completed.cached_payload = provider_health_cache.get(health_cache_key)- if cached_payload is not None:+ if cached_payload is not None and not has_real_failure:logger.debug("Returning cached provider-health response (waited for in-flight)")return JSONResponse(cached_payload, status_code=200)# Leader's validation failed; retry leader election rather than@@ -268,6 +278,19 @@ async def check_provider_health(f"Embedding provider ({embedding_provider}) validation failed: {embedding_error}")+ # A real call beats a probe. The probe sends its own request, so it+ # hits its own failure: OpenAI checks request shape before billing,+ # which is how a probe can report "no credits remaining" while the+ # agent's own call reports a 400 about its parameters. Both are+ # true; the actionable one is the one the user's traffic produced.+ # An entry only exists while calls are still failing — the gateway+ # erases it on the next success.+ llm_error = provider_error_log.latest_failure(provider, "chat") or llm_error+ embedding_error = (+ provider_error_log.latest_failure(embedding_provider, "embedding")+ or embedding_error+ )+# Return combined statusif llm_error or embedding_error:errors = []diff --git a/src/api/settings/helpers.py b/src/api/settings/helpers.pyindex 3b31b2ace..a6f04071d 100644--- a/src/api/settings/helpers.py+++ b/src/api/settings/helpers.py@@ -43,26 +43,41 @@ def _configured_provider_names(config, provider_names) -> list:def _first_configured_llm_provider(config, excluding: str) -> str:- """Return the first configured LLM provider that isn't `excluding`."""+ """Return the first configured LLM provider that isn't `excluding`.++ Only providers this run mode shows are eligible: falling back to one the+ config file hides would leave Settings pointing at a provider whose card+ the user cannot see (see ``config/model_providers.yaml``).+ """+ from config.model_providers import visible_provider_keys++ visible = visible_provider_keys()for p in _LLM_PROVIDER_NAMES:- if p != excluding and getattr(config.providers, p).configured:+ if p != excluding and p in visible and getattr(config.providers, p).configured:return pfor provider, value in config.providers.custom.items():- if provider != excluding and value.configured:+ if provider != excluding and provider in visible and value.configured:return providerreturn "openai"def _first_configured_embedding_provider(config, excluding: str) -> str:- """Return the first configured embedding provider that isn't `excluding`, or "" if none."""+ """Return the first configured embedding provider that isn't `excluding`, or "" if none.++ Providers hidden in this run mode are skipped, as in+ ``_first_configured_llm_provider``.+ """+ from config.model_providers import visible_provider_keys++ visible = visible_provider_keys()for p in _EMBEDDING_PROVIDER_NAMES:- if p != excluding and getattr(config.providers, p).configured:+ if p != excluding and p in visible and getattr(config.providers, p).configured:return pfrom services.model_catalog import catalogentries = {entry["key"]: entry for entry in catalog()["providers"]}for provider, value in config.providers.custom.items():- if provider != excluding and value.configured:+ if provider != excluding and provider in visible and value.configured:entry = entries.get(provider)if entry and entry["embedding_models"]:return providerdiff --git a/src/api/v1/llm.py b/src/api/v1/llm.pyindex 63053eb2e..d7373fc9e 100644--- a/src/api/v1/llm.py+++ b/src/api/v1/llm.py@@ -58,13 +58,39 @@ async def model_catalog_endpoint(user: User = Depends(_catalog_read),):"""LiteLLM picker payload. GET /v1/model-catalog"""+ # `no-store` for the same reason as /models/providers: the catalogue is+ # filtered by that config file, so a cached copy survives the restart that+ # was supposed to apply the edit. LiteLLM's table is cached in-process, so+ # rebuilding this response is cheap.try:- return JSONResponse(catalog(), headers={"Cache-Control": "private, max-age=3600"})+ return JSONResponse(catalog(), headers={"Cache-Control": "no-store"})except CatalogUnavailableError as exc:logger.error("Model catalogue unavailable", error=str(exc))return JSONResponse({"error": CATALOG_UNAVAILABLE_MESSAGE}, status_code=503)+async def model_providers_endpoint(+ user: User = Depends(_catalog_read),+):+ """Providers this run mode exposes. GET /v1/model-providers++ Same list the console reads from /models/providers, so an SDK client and the+ UI never disagree about which providers a deployment offers.+ """+ from config.model_providers import provider_visibility_payload++ # `no-store`, not a max-age: the list is derived from+ # `config/model_providers.yaml`, which an operator edits and then restarts+ # the backend for. A cached response outlives that restart in the browser,+ # so the console keeps drawing the old provider cards/tabs and the change+ # looks like it did nothing. The payload is a few hundred bytes and React+ # Query already holds it for the session, so the round trip costs nothing.+ return JSONResponse(+ provider_visibility_payload(),+ headers={"Cache-Control": "no-store"},+ )++async def _read_json_body(request: Request) -> dict[str, Any] | JSONResponse:try:body = await request.json()diff --git a/src/app/routes/internal.py b/src/app/routes/internal.pyindex 70d454ec4..1fcec091c 100644--- a/src/app/routes/internal.py+++ b/src/app/routes/internal.py@@ -439,6 +439,9 @@ def register_internal_routes(app: FastAPI):app.add_api_route("/models/catalog", models.get_model_catalog, methods=["GET"], tags=["internal"])+ app.add_api_route(+ "/models/providers", models.get_model_providers, methods=["GET"], tags=["internal"]+ )# Onboarding endpointsapp.add_api_route("/onboarding", settings.onboarding, methods=["POST"], tags=["internal"])diff --git a/src/app/routes/public_v1.py b/src/app/routes/public_v1.pyindex 690c8e8ef..5b54390ad 100644--- a/src/app/routes/public_v1.py+++ b/src/app/routes/public_v1.py@@ -112,6 +112,12 @@ def register_public_v1_routes(app: FastAPI):methods=["GET"],tags=["public"],)+ app.add_api_route(+ "/v1/model-providers",+ v1_llm.model_providers_endpoint,+ methods=["GET"],+ tags=["public"],+ )app.add_api_route("/v1/chat/completions",v1_llm.chat_completions_endpoint,diff --git a/src/config/model_providers.py b/src/config/model_providers.pynew file mode 100644index 000000000..d96adaeb6--- /dev/null+++ b/src/config/model_providers.py@@ -0,0 +1,277 @@+"""Which model providers this deployment offers, per run mode.++``model_providers.yaml`` (next to this module) lists every provider OpenRAG can+show and, for each one, whether it is visible in ``oss``, ``on_prem`` and+``saas``. The backend filters that list by ``OPENRAG_RUN_MODE`` and serves the+result; Settings cards, the Onboarding tabs and the model catalogue all read+that one list, so per-deployment policy is a config row rather than a denylist+scattered across the frontend.++Two rules keep the file safe to extend:++- a mode key that is missing (or not a boolean-ish value) means *hidden*, so a+ new run mode cannot silently expose a provider;+- a provider absent from the file is not offered anywhere, though a model id+ for it can still be typed by hand if the gateway already routes it.++A row may also declare `models` / `embedding_models`. LiteLLM's bundled table+covers the public vendors, but a self-hosted OpenAI-compatible gateway serves+whatever its operator deployed, so those ids can only come from here. Declared+ids are added to whatever the table already knows for that provider.++``OPENRAG_MODEL_PROVIDERS_CONFIG`` points at an alternate YAML file for a single+deployment. If that file is missing or unreadable the shipped default is used;+if the shipped default is unreadable too, ``_FALLBACK_PROVIDERS`` below keeps+the process serving the pre-config behavior instead of hiding every provider.+"""++from __future__ import annotations++import os+from functools import lru_cache+from pathlib import Path+from typing import Any, NamedTuple++import yaml++from utils.logging_config import get_logger+from utils.run_mode_utils import RUN_MODE_ON_PREM, RUN_MODE_OSS, RUN_MODE_SAAS, get_run_mode++logger = get_logger(__name__)++CONFIG_PATH_ENV = "OPENRAG_MODEL_PROVIDERS_CONFIG"+DEFAULT_CONFIG_PATH = Path(__file__).with_name("model_providers.yaml")++KNOWN_RUN_MODES: tuple[str, ...] = (RUN_MODE_OSS, RUN_MODE_ON_PREM, RUN_MODE_SAAS)++# Last-resort table, used only when no YAML file can be read at all. It matches+# the visibility OpenRAG shipped before this config existed.+_FALLBACK_PROVIDERS: tuple[dict[str, Any], ...] = (+ {+ "name": "openai",+ "display_name": "OpenAI",+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: True},+ "models": (),+ "embedding_models": (),+ "exclude_models": (),+ },+ {+ "name": "ollama",+ "display_name": "Ollama",+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: False},+ "models": (),+ "embedding_models": (),+ "exclude_models": (),+ },+ {+ "name": "watsonx",+ "display_name": "IBM watsonx.ai",+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: True},+ "models": (),+ "embedding_models": (),+ "exclude_models": (),+ },+ {+ "name": "anthropic",+ "display_name": "Anthropic",+ "modes": {RUN_MODE_OSS: True, RUN_MODE_ON_PREM: True, RUN_MODE_SAAS: True},+ "models": (),+ "embedding_models": (),+ "exclude_models": (),+ },+)++_TRUTHY = {"true", "1", "yes", "on"}+++def _as_bool(value: Any) -> bool:+ """Whether a YAML `modes` value means "visible".++ Anything that is not a boolean or a recognised truthy string is False, so a+ typo hides a provider rather than exposing it.+ """+ if isinstance(value, bool):+ return value+ if isinstance(value, str):+ return value.strip().lower() in _TRUTHY+ return False+++def _model_ids(value: Any) -> tuple[str, ...]:+ """A YAML `models:` list as clean, de-duplicated, order-preserving ids."""+ if not isinstance(value, list):+ return ()+ ids = []+ for item in value:+ name = str(item).strip()+ if name and name not in ids:+ ids.append(name)+ return tuple(ids)+++def _model_patterns(value: Any) -> tuple[str, ...]:+ """A YAML `exclude_models:` list as lowercased fnmatch patterns."""+ if not isinstance(value, list):+ return ()+ patterns = []+ for item in value:+ pattern = str(item).strip().lower()+ if pattern and pattern not in patterns:+ patterns.append(pattern)+ return tuple(patterns)+++def _normalize(entry: Any, seen: set[str]) -> dict[str, Any] | None:+ """One YAML provider row as a validated dict, or None if unusable."""+ if not isinstance(entry, dict):+ logger.warning("Ignoring a model provider entry that is not a mapping")+ return None+ name = str(entry.get("name") or "").strip().lower()+ if not name:+ logger.warning("Ignoring a model provider entry with no name")+ return None+ if name in seen:+ logger.warning("Ignoring a duplicate model provider entry", provider=name)+ return None+ seen.add(name)+ raw_modes = entry.get("modes")+ if not isinstance(raw_modes, dict):+ raw_modes = {}+ display_name = str(entry.get("display_name") or "").strip() or name+ return {+ "name": name,+ "display_name": display_name,+ # Missing keys stay missing here; `is_visible` reads them as False.+ "modes": {str(mode).strip().lower(): _as_bool(value) for mode, value in raw_modes.items()},+ # Optional: ids the catalogue cannot learn from LiteLLM's static table.+ "models": _model_ids(entry.get("models")),+ "embedding_models": _model_ids(entry.get("embedding_models")),+ # Optional: ids to keep out of the pickers, `*`/`?` wildcards allowed.+ "exclude_models": _model_patterns(entry.get("exclude_models")),+ }+++def _parse(raw: Any, source: str) -> tuple[dict[str, Any], ...]:+ if isinstance(raw, dict):+ entries = raw.get("providers")+ else:+ entries = raw+ if not isinstance(entries, list):+ logger.error("Model providers config has no `providers` list", source=source)+ return ()+ seen: set[str] = set()+ parsed = [_normalize(entry, seen) for entry in entries]+ return tuple(entry for entry in parsed if entry is not None)+++def _read(path: Path) -> tuple[dict[str, Any], ...]:+ try:+ raw = yaml.safe_load(path.read_text(encoding="utf-8"))+ except FileNotFoundError:+ logger.warning("Model providers config not found", path=str(path))+ return ()+ except (OSError, yaml.YAMLError) as exc:+ logger.error(+ "Model providers config could not be read",+ path=str(path),+ error=str(exc),+ )+ return ()+ return _parse(raw, str(path))+++@lru_cache(maxsize=1)+def _configured(override: str, default_path: str) -> tuple[dict[str, Any], ...]:+ """Every provider in the config file, in file order, mode filtering aside.++ Cached on the resolved paths so the YAML is read once per process. Call+ `reload()` after changing the file or the override env var.+ """+ if override:+ providers = _read(Path(override))+ if providers:+ return providers+ logger.error(+ "Falling back to the shipped model providers config",+ override=override,+ env=CONFIG_PATH_ENV,+ )+ providers = _read(Path(default_path))+ if providers:+ return providers+ logger.error(+ "No usable model providers config; using built-in defaults",+ path=default_path,+ )+ return _FALLBACK_PROVIDERS+++def configured_providers() -> tuple[dict[str, Any], ...]:+ """Every provider the config file lists, regardless of run mode."""+ return _configured((os.getenv(CONFIG_PATH_ENV) or "").strip(), str(DEFAULT_CONFIG_PATH))+++def reload() -> None:+ """Drop the cached file so the next read re-parses it."""+ _configured.cache_clear()+++def is_visible(provider: dict[str, Any], run_mode: str) -> bool:+ """Whether a parsed provider entry is shown in `run_mode`."""+ return bool(provider["modes"].get(run_mode, False))+++def visible_providers(run_mode: str | None = None) -> tuple[dict[str, Any], ...]:+ """The providers shown in `run_mode` (the current one by default)."""+ mode = (run_mode or get_run_mode()).strip().lower()+ return tuple(entry for entry in configured_providers() if is_visible(entry, mode))+++class ProviderEntry(NamedTuple):+ """One visible provider, in the form the catalogue consumes.++ A NamedTuple rather than a dict: hashable all the way down, so callers can+ key an lru_cache on a tuple of these and have a config change rebuild it.+ """++ name: str+ display_name: str+ models: tuple[str, ...]+ embedding_models: tuple[str, ...]+ exclude_models: tuple[str, ...]+++def visible_provider_entries(+ run_mode: str | None = None,+) -> tuple[ProviderEntry, ...]:+ """The visible providers, as catalogue-ready entries."""+ return tuple(+ ProviderEntry(+ entry["name"],+ entry["display_name"],+ entry["models"],+ entry["embedding_models"],+ entry["exclude_models"],+ )+ for entry in visible_providers(run_mode)+ )+++def visible_provider_keys(run_mode: str | None = None) -> frozenset[str]:+ return frozenset(entry["name"] for entry in visible_providers(run_mode))+++def is_provider_visible(provider: str, run_mode: str | None = None) -> bool:+ return (provider or "").strip().lower() in visible_provider_keys(run_mode)+++def provider_visibility_payload(run_mode: str | None = None) -> dict[str, Any]:+ """The API body: the current run mode and the providers it exposes."""+ mode = (run_mode or get_run_mode()).strip().lower()+ return {+ "run_mode": mode,+ "providers": [+ {"name": entry["name"], "display_name": entry["display_name"]}+ for entry in visible_providers(mode)+ ],+ }diff --git a/src/config/model_providers.yaml b/src/config/model_providers.yamlnew file mode 100644index 000000000..549c55de1--- /dev/null+++ b/src/config/model_providers.yaml@@ -0,0 +1,114 @@+# Which model providers OpenRAG offers, per run mode.+#+# This file is the single source of truth for provider visibility in the+# Settings provider cards, the Onboarding tabs, and the model catalogue that+# feeds every agent/ingest picker. The backend filters on it; the UI renders+# whatever the API returns, so there is no second denylist in the frontend.+#+# Keys under `modes` match OPENRAG_RUN_MODE: oss | on_prem | saas.+# true -> the provider is shown in that mode+# false -> the provider is hidden+# A missing mode key means hidden, so adding a new run mode can never expose a+# provider by accident.+#+# A provider that is not listed here is not offered anywhere in the UI. A model+# id for an unlisted provider can still be typed by hand and will be routed if+# the gateway already supports it.+#+# Set OPENRAG_MODEL_PROVIDERS_CONFIG to another YAML file to override this list+# for a single deployment; this file is the default.+#+# A row may also declare `models` / `embedding_models`. LiteLLM's bundled table+# already knows the public vendors, so those rows need nothing. A self-hosted+# OpenAI-compatible gateway serves whatever its operator deployed, and those ids+# can only come from here — see the commented example at the end of this file.+#+# `exclude_models` keeps ids out of the pickers. `*` and `?` wildcards work, and+# matching is against the id as the picker shows it (`gpt-3.5-turbo`, not+# `openai/gpt-3.5-turbo`). Exclusion is applied last, so it also overrides a+# `models` row on the same provider. Models the provider has already retired are+# dropped without being listed here, and ones retiring within 90 days are hidden+# behind a "Show N retiring" row in the picker — use this for the ones a+# deployment does not want offered at all.++providers:+ - name: openai+ display_name: OpenAI+ modes:+ oss: true+ on_prem: true+ saas: true+ exclude_models:+ - gpt-3.5-* # a whole generation, superseded and near retirement+ - gpt-5.6-luna # refuses function tools on /v1/chat/completions++ - name: ollama+ display_name: Ollama+ modes:+ oss: true+ on_prem: true+ saas: false # local inference is not offered in SaaS++ - name: watsonx+ display_name: IBM watsonx.ai+ modes:+ oss: true+ on_prem: true+ saas: true++ - name: anthropic+ display_name: Anthropic+ modes:+ oss: true+ on_prem: true+ saas: true+ # enabled in OSS for tests only to be disbaled in OSS after tests are complete+ - name: azure_ai+ display_name: Azure AI Foundry+ modes:+ oss: false+ on_prem: false+ saas: false+ # enabled in OSS for tests only to be disbaled in OSS after tests are complete++ # Azure OpenAI Service, which LiteLLM keys separately from Foundry above:+ # `azure_ai` is the Foundry catalogue (Llama, Phi, Mistral, Cohere, and the+ # gpt-5.x entries Foundry serves), `azure` is the OpenAI Service deployment+ # where the gpt-4.1 family lives. Both can be offered at once.+ - name: azure+ display_name: Azure OpenAI+ modes:+ oss: true+ on_prem: true+ saas: true+ # Azure serves the same OpenAI models, so it needs the same exclusions.+ # The plain name covers the regional ids (`eu/`, `us/`, `global/`) too.+ exclude_models:+ - gpt-3.5-*+ # Azure lets you name a deployment anything, so LiteLLM's table cannot know+ # yours. Declare them here and they appear in the pickers for everyone;+ # otherwise each user has to type the name into the model box by hand.+ # models:+ # - prod-gpt41-eastus+ # embedding_models:+ # - prod-embed-3-large++ # A self-hosted OpenAI-compatible endpoint (vLLM, LM Studio, an internal+ # gateway). Use one of LiteLLM's passthrough provider keys — `openai_like`,+ # `hosted_vllm`, `lm_studio`, `llamafile`, `custom_openai` — rather than+ # inventing a name: the key has to be one LiteLLM can route, or the+ # `provider:model` ids it publishes resolve to the default provider and get+ # called with the wrong credentials. The endpoint and key are entered in+ # Settings; only the model ids belong here.+ #+ # - name: openai_like+ # display_name: Internal LLM Gateway+ # modes:+ # oss: true+ # on_prem: true+ # saas: false+ # models:+ # - llama-3.3-70b-instruct+ # - qwen2.5-coder-32b+ # embedding_models:+ # - bge-m3diff --git a/src/mcp_http/server.py b/src/mcp_http/server.pyindex 09b5d53a4..35503195c 100644--- a/src/mcp_http/server.py+++ b/src/mcp_http/server.py@@ -171,6 +171,13 @@ COMPONENT_CUSTOMIZATIONS: dict[tuple[str, str], dict[str, str]] = {"grouped models, capability flags, and credential field specs."),},+ ("/v1/model-providers", "GET"): {+ "name": "openrag_model_providers",+ "description": (+ "Model providers this deployment exposes for its run mode "+ "(oss / on_prem / saas), with their display names."+ ),+ },("/v1/chat/completions", "POST"): {"name": "openrag_chat_completions","description": (diff --git a/src/services/llm_gateway.py b/src/services/llm_gateway.pyindex 5d5a8f236..dffa73917 100644--- a/src/services/llm_gateway.py+++ b/src/services/llm_gateway.py@@ -12,6 +12,7 @@ import refrom collections.abc import AsyncIterator, Mappingfrom typing import Any, Literal+from services import provider_error_logfrom services.model_catalog import is_known_providerfrom utils.logging_config import get_logger@@ -504,6 +505,53 @@ def _log_completion_shape(payload: dict[str, Any], provider: str, model: str) ->logger.debug("Could not summarise completion shape", exc_info=True)+#: Models seen to refuse function tools alongside their own default reasoning+#: effort. OpenAI applies a non-none default to reasoning models and rejects+#: that combination on /v1/chat/completions; the remedy its error names is an+#: explicit "none". Learned from the first failure rather than assumed, so a+#: model that accepts the pair keeps its reasoning.+_TOOLS_NEED_REASONING_OFF: set[str] = set()+++def _model_info(model: str) -> dict[str, Any]:+ """LiteLLM's row for `model`, by its full id or its bare name."""+ try:+ import litellm++ table = litellm.model_cost+ info = table.get(model) or table.get(model.rsplit("/", 1)[-1])+ return info if isinstance(info, dict) else {}+ except Exception:+ return {}+++def _is_reasoning_tool_conflict(detail: str) -> bool:+ """Whether a failure is the tools-plus-reasoning_effort rejection.++ Matched on the provider's own words. There is no capability flag for it —+ `supports_reasoning` and `supports_function_calling` are both true for the+ models that refuse the combination — so the failure itself is the only+ signal available.+ """+ lowered = (detail or "").lower()+ return "reasoning_effort" in lowered and "tool" in lowered+++def _reasoning_off_retry(model: str, kwargs: dict[str, Any]) -> bool:+ """Set `reasoning_effort="none"` for a retry, if that can help here.++ False when the caller chose an effort itself — overriding a deliberate+ choice would be worse than the error — or when the model cannot take+ "none", in which case only the Responses API can serve tools for it.+ """+ if not kwargs.get("tools") or "reasoning_effort" in kwargs:+ return False+ if not _model_info(model).get("supports_none_reasoning_effort"):+ return False+ kwargs["reasoning_effort"] = "none"+ return True++async def chat_completions(body: Mapping[str, Any], *, config=None) -> dict[str, Any] | AsyncIterator[str]:@@ -512,10 +560,14 @@ async def chat_completions(litellm_model, provider, credentials = resolve_call(body.get("model"), kind="chat", config=cfg)kwargs = {key: body[key] for key in _LITELLM_FORWARDED_PARAMS if key in body}stream = bool(body.get("stream"))- try:+ if litellm_model in _TOOLS_NEED_REASONING_OFF:+ # Already learned about this model; do not spend a round-trip relearning.+ _reasoning_off_retry(litellm_model, kwargs)++ async def _call() -> Any:import litellm- result = await litellm.acompletion(+ return await litellm.acompletion(model=litellm_model,messages=list(body.get("messages") or []),stream=stream,@@ -529,6 +581,25 @@ async def chat_completions(**credentials,**kwargs,)++ try:+ try:+ result = await _call()+ except Exception as exc:+ # `drop_params` cannot help here: reasoning_effort is a supported+ # parameter, just not one this model accepts beside tools. Retry+ # once with the value the provider's own error asks for.+ if not _is_reasoning_tool_conflict(f"{exc}") or not _reasoning_off_retry(+ litellm_model, kwargs+ ):+ raise+ logger.info(+ "Retrying completion with reasoning_effort=none",+ provider=provider,+ model=litellm_model,+ )+ _TOOLS_NEED_REASONING_OFF.add(litellm_model)+ result = await _call()except LlmGatewayError:raiseexcept Exception as exc:@@ -536,14 +607,22 @@ async def chat_completions(logger.error("LLM chat completions failed", provider=provider, model=litellm_model, error=detail)+ message = _upstream_client_message(detail, provider, litellm_model, exc)+ # The health banner otherwise reports whatever its own probe hit, which+ # is a different request and so often a different error. Hand it the+ # text this caller is being shown.+ provider_error_log.record_failure(provider, "chat", message)raise LlmGatewayError(- _upstream_client_message(detail, provider, litellm_model, exc),+ message,_upstream_status_code(exc),detail=detail,) from excif stream:+ # A stream can still fail mid-flight, so _stream_sse clears the record+ # itself once the provider has actually delivered something.return _stream_sse(result, provider, litellm_model, credentials)+ provider_error_log.record_success(provider, "chat")payload = _to_openai_dict(result)_repair_completion_payload(payload, provider, litellm_model)_log_completion_shape(payload, provider, litellm_model)@@ -797,6 +876,9 @@ async def _stream_sse(# the next person has something to search for.if tally.content_chars == 0 and tally.tool_calls == 0:tally.error = "empty completion"+ provider_error_log.record_failure(+ provider, "chat", _empty_stream_message(provider, model, tally.finish_reason)+ )logger.warning("LLM chat stream produced no content and no tool calls",provider=provider,@@ -807,6 +889,10 @@ async def _stream_sse(yield _error_frame(_empty_stream_message(provider, model, tally.finish_reason), provider, model)+ else:+ # Content actually reached the client, so whatever the banner was+ # holding against this provider is no longer true.+ provider_error_log.record_success(provider, "chat")except Exception as exc:# Mid-stream failures used to surface as a truncated stream, which the# UI could only report as "the server didn't return a response". Emit an@@ -815,7 +901,9 @@ async def _stream_sse(detail = _redact(f"{type(exc).__name__}: {exc}", credentials or {})tally.error = detaillogger.error("LLM chat stream failed", provider=provider, model=model, error=detail)- yield _error_frame(_upstream_client_message(detail, provider, model, exc), provider, model)+ message = _upstream_client_message(detail, provider, model, exc)+ provider_error_log.record_failure(provider, "chat", message)+ yield _error_frame(message, provider, model)finally:close = getattr(stream, "aclose", None) or getattr(stream, "close", None)if close is not None:@@ -866,9 +954,12 @@ async def embeddings(body: Mapping[str, Any], *, config=None) -> dict[str, Any]:except Exception as exc:detail = _redact(f"{type(exc).__name__}: {exc}", credentials)logger.error("LLM embeddings failed", provider=provider, model=litellm_model, error=detail)+ message = _upstream_client_message(detail, provider, litellm_model, exc)+ provider_error_log.record_failure(provider, "embedding", message)raise LlmGatewayError(- _upstream_client_message(detail, provider, litellm_model, exc),+ message,_upstream_status_code(exc),detail=detail,) from exc+ provider_error_log.record_success(provider, "embedding")return _to_openai_dict(result)diff --git a/src/services/model_catalog.py b/src/services/model_catalog.pyindex c1da80405..47a4d074f 100644--- a/src/services/model_catalog.py+++ b/src/services/model_catalog.py@@ -16,25 +16,31 @@ JSON is ~100KB, so paying for it once per process beats a re-read per request.`function_calling` and `vision` are published as capability flags so the UI canfilter (agent vs VLM) without a second live provider call.-The public catalogue is intentionally limited to the providers OpenRAG exposes-in its settings UI. Generic credential helpers remain available for future-providers, and a model that is not in the catalogue can still be typed by hand.+The public catalogue is limited to the providers this run mode exposes, per+``config/model_providers.yaml`` (see ``config.model_providers``). Generic+credential helpers remain available for every other provider, and a model that+is not in the catalogue can still be typed by hand."""from __future__ import annotationsimport datetimeimport json+from fnmatch import fnmatchfrom functools import lru_cachefrom typing import Any+from config.model_providers import ProviderEntry, visible_provider_entriesfrom utils.logging_config import get_loggerlogger = get_logger(__name__)TEXT_GENERATION_MODES = frozenset({"chat", "completion", "responses"})EMBEDDING_MODE = "embedding"-SUPPORTED_PROVIDER_KEYS = frozenset({"openai", "anthropic", "watsonx", "ollama"})++#: LiteLLM prices fine-tunes off a template row named for the base model. The+#: row is not a callable id, so it never belongs in a picker.+FINE_TUNE_TEMPLATE_PREFIX = "ft:"CAPABILITY_FLAGS: dict[str, str] = {"supports_function_calling": "function_calling",@@ -192,14 +198,50 @@ def _model_entry(name: str, info: dict[str, Any]) -> dict[str, Any]:return entry-@lru_cache(maxsize=1)-def _catalog() -> dict[str, Any]:+def _excluded(name: str, patterns: tuple[str, ...]) -> bool:+ """Whether a model id is one the config file keeps out of the pickers.++ Matched on the id as the picker shows it — `gpt-3.5-turbo`, not+ `openai/gpt-3.5-turbo` — because that is the name an operator reads off the+ screen when deciding what to suppress. `*` and `?` work, so a whole+ generation goes in one line.++ Also matched on the id's last path segment, so a regional deployment is+ covered by the plain name: Azure lists `gpt-5.6-luna` as `eu/gpt-5.6-luna`+ and `us/gpt-5.6-luna` too, and someone suppressing that model means all+ three. Write `eu/*` to reach a region on its own.+ """+ if not patterns:+ return False+ lowered = name.lower()+ bare = lowered.rsplit("/", 1)[-1]+ return any(fnmatch(lowered, pattern) or fnmatch(bare, pattern) for pattern in patterns)+++def _declared_entries(names: tuple[str, ...], mode: str) -> list[dict[str, Any]]:+ """Config-declared model ids as catalogue entries.++ A self-hosted OpenAI-compatible gateway serves whatever its operator+ deployed, so LiteLLM's bundled table knows none of its ids. Only the name+ and the mode are known here — no pricing, no capability flags.+ """+ return [{"model": name, "mode": mode} for name in names]+++@lru_cache(maxsize=8)+def _catalog(providers: tuple[ProviderEntry, ...]) -> dict[str, Any]:+ """Payload for `providers`, given as `config.model_providers` entries.++ The entries are the cache key, so flipping a provider's visibility (or the+ run mode, in a test) rebuilds instead of serving a stale list.+ """try:import litellmexcept Exception as exc:raise CatalogUnavailableError("litellm is not installed on the server") from excspecs = _provider_field_specs()+ keys = {entry.name for entry in providers}chat_by_provider: dict[str, list[dict[str, Any]]] = {}embed_by_provider: dict[str, list[dict[str, Any]]] = {}@@ -208,7 +250,7 @@ def _catalog() -> dict[str, Any]:continueprovider = info.get("litellm_provider")mode = info.get("mode")- if provider not in SUPPORTED_PROVIDER_KEYS:+ if provider not in keys:continueif mode in TEXT_GENERATION_MODES:bucket = chat_by_provider@@ -219,36 +261,66 @@ def _catalog() -> dict[str, Any]:name = model_id[len(provider) + 1 :] if model_id.startswith(f"{provider}/") else model_idif not name or name == "sample_spec":continue+ if name.startswith(FINE_TUNE_TEMPLATE_PREFIX):+ # `ft:gpt-4o-2024-08-06` is a pricing row for the *base* of a+ # fine-tune, not an id anyone can call: a real one carries the org+ # and job suffix (`ft:gpt-4o-2024-08-06:acme::abc123`). Listing the+ # template invites picking a model that 404s. Owners of a fine-tune+ # type their full id into the picker instead.+ continuebucket.setdefault(provider, []).append(_model_entry(name, info))- providers = []- for key in sorted(SUPPORTED_PROVIDER_KEYS):- providers.append(+ entries = []+ for key, display_name, declared_chat, declared_embed, excluded in providers:+ if not is_known_provider(key):+ # The catalogue would still render, but `split_model_id` cannot+ # recognise the prefix, so every id would be billed to the default+ # provider. Name the row rather than fail the whole catalogue.+ logger.warning(+ "Model provider is not routable by LiteLLM; its models will not resolve",+ provider=key,+ )+ chat = chat_by_provider.get(key, [])+ embed = embed_by_provider.get(key, [])+ known_chat = {entry["model"] for entry in chat}+ known_embed = {entry["model"] for entry in embed}+ chat = chat + _declared_entries(+ tuple(name for name in declared_chat if name not in known_chat), "chat"+ )+ embed = embed + _declared_entries(+ tuple(name for name in declared_embed if name not in known_embed), EMBEDDING_MODE+ )+ # Applied last, so `exclude_models` also wins over a `models:` row —+ # a deployment that suppresses an id means it, wherever the id came+ # from, and the alternative is two lines that quietly contradict.+ chat = [entry for entry in chat if not _excluded(entry["model"], excluded)]+ embed = [entry for entry in embed if not _excluded(entry["model"], excluded)]+ entries.append({"key": key,- "name": (specs.get(key) or {}).get("provider_display_name") or key,+ # The config file names the provider; LiteLLM's own label is+ # the fallback for a row that left `display_name` out.+ "name": display_name or (specs.get(key) or {}).get("provider_display_name") or key,"credential_fields": credential_fields(key),"model_placeholder": (specs.get(key) or {}).get("default_model_placeholder"),- "models": sorted(chat_by_provider.get(key, []), key=lambda entry: entry["model"]),- "embedding_models": sorted(- embed_by_provider.get(key, []), key=lambda entry: entry["model"]- ),+ "models": sorted(chat, key=lambda entry: entry["model"]),+ "embedding_models": sorted(embed, key=lambda entry: entry["model"]),})- return {"providers": providers}+ return {"providers": entries}-@lru_cache(maxsize=2)-def _catalog_for(today: datetime.date) -> dict[str, Any]:+@lru_cache(maxsize=8)+def _catalog_for(today: datetime.date, providers: tuple[ProviderEntry, ...]) -> dict[str, Any]:"""`_catalog()` with models the provider has already retired removed."""def _keep(model: dict[str, Any]) -> bool:retires = _parse_deprecation(model.get("deprecation_date"))return retires is None or retires > today- providers = []- for provider in _catalog()["providers"]:- providers.append(+ entries = []+ for provider in _catalog(providers)["providers"]:+ entries.append({**provider,"models": [model for model in provider["models"] if _keep(model)],@@ -257,12 +329,56 @@ def _catalog_for(today: datetime.date) -> dict[str, Any]:],})- return {"providers": providers}+ return {"providers": entries}+++def exclusions_for(provider: str) -> tuple[str, ...]:+ """The `exclude_models` patterns configured for `provider`."""+ key = (provider or "").strip().lower()+ for entry in visible_provider_entries():+ if entry.name == key:+ return entry.exclude_models+ return ()+++#: Keys in a live `/models/{provider}` payload that hold model lists.+LIVE_MODEL_LIST_KEYS = ("language_models", "embedding_models")+++def hide_excluded_live_models(provider: str, payload: Any) -> Any:+ """Apply `exclude_models` to a live `/models/{provider}` response.++ Ollama, watsonx, OpenAI and Anthropic are also listed by asking the running+ provider, which never passes through the catalogue — so without this an+ excluded id disappears from the settings picker and comes straight back in+ through onboarding. The filter belongs here rather than in the frontend so+ every consumer, SDK clients included, sees one list.+ """+ patterns = exclusions_for(provider)+ if not patterns or not isinstance(payload, dict):+ return payload+ filtered = dict(payload)+ for key in LIVE_MODEL_LIST_KEYS:+ models = payload.get(key)+ if not isinstance(models, list):+ continue+ filtered[key] = [+ model+ for model in models+ if not _excluded(str((model or {}).get("value", "")), patterns)+ if isinstance(model, dict)+ ]+ return filtered+++def supported_provider_keys() -> frozenset[str]:+ """The providers this run mode publishes, per `config/model_providers.yaml`."""+ return frozenset(entry.name for entry in visible_provider_entries())def catalog(today: datetime.date | None = None) -> dict[str, Any]:- """Picker payload for OpenRAG's supported providers and their models."""- return _catalog_for(today or datetime.date.today())+ """Picker payload for the providers this run mode exposes, and their models."""+ return _catalog_for(today or datetime.date.today(), visible_provider_entries())def is_known_provider(provider: str) -> bool:@@ -273,7 +389,14 @@ def is_known_provider(provider: str) -> bool:try:import litellm- if key in {str(value) for value in litellm.provider_list}:+ # `provider_list` holds LlmProviders enum members, and str() on one+ # yields "LlmProviders.OPENAI" — never the routable "openai". Reading+ # .value is what makes this branch match; without it the check fell+ # through to the credential-form specs below, which do not cover every+ # provider LiteLLM can route (zai, scaleway, chatgpt, ...). Those+ # prefixes were then left unsplit and called with the default+ # provider's credentials.+ if key in {getattr(value, "value", str(value)) for value in litellm.provider_list}:return Trueexcept Exception:logger.debug("Could not read litellm.provider_list", exc_info=True)@@ -283,15 +406,19 @@ def is_known_provider(provider: str) -> bool:return False-@lru_cache(maxsize=1)-def _model_owners() -> dict[str, tuple[str, ...]]:- """Map every catalogue model id to the providers that serve it."""+@lru_cache(maxsize=8)+def _model_owners(providers: tuple[ProviderEntry, ...]) -> dict[str, tuple[str, ...]]:+ """Map every catalogue model id to the providers that serve it.++ Keyed by the same entries as `_catalog()`, so a run mode that hides a+ provider also drops that provider's models from the ownership map.+ """owners: dict[str, set[str]] = {}try:- providers = _catalog()["providers"]+ entries = _catalog(providers)["providers"]except Exception:return {}- for provider in providers:+ for provider in entries:for entry in (*provider["models"], *provider["embedding_models"]):owners.setdefault(entry["model"], set()).add(provider["key"])return {model: tuple(sorted(keys)) for model, keys in owners.items()}@@ -307,7 +434,7 @@ def catalog_owner(model: str) -> str | None:if not name:return Nonetry:- owners = _model_owners().get(name)+ owners = _model_owners(visible_provider_entries()).get(name)except Exception:return Nonereturn owners[0] if owners and len(owners) == 1 else None@@ -320,7 +447,7 @@ def PROVIDER_SEPARATOR_SAFE_CHECK() -> list[str]:Exposed so a test can fail loudly if a future model id breaks the property."""ambiguous = []- for model in _model_owners():+ for model in _model_owners(visible_provider_entries()):head, sep, rest = model.partition(":")if sep and rest and is_known_provider(head.lower()):ambiguous.append(model)diff --git a/src/services/provider_error_log.py b/src/services/provider_error_log.pynew file mode 100644index 000000000..424ffe481--- /dev/null+++ b/src/services/provider_error_log.py@@ -0,0 +1,73 @@+"""The most recent real provider failure, per provider and call kind.++The health banner reports what a synthetic probe hit, and a probe cannot+reproduce the request that actually failed. OpenAI validates request shape+before it checks billing, so a plain probe completion comes back "You have no+credits remaining" while the agent's own call — function tools plus+``reasoning_effort`` — comes back a 400 about the parameters. Both are true.+The one the operator needs is the one their own traffic produced.++So the gateway records what it saw. An entry is written when a call fails and+erased when the next call on the same provider and kind succeeds, which means+its presence says traffic is failing *now* rather than that it once did.+``STALE_AFTER_SECONDS`` covers the remaining case: a provider that fails and+is then never called again would otherwise pin the banner forever.++In-process and per-worker, deliberately. This is a diagnostic for the operator+watching the console, not a durable record — an entry that is lost on restart+is one whose failure has not recurred.+"""++from __future__ import annotations++import time+from typing import Literal++CallKind = Literal["chat", "embedding"]++#: How long a recorded failure keeps speaking for a provider that has gone+#: quiet. Long enough to survive the banner's polling gap, short enough that a+#: failure nobody is reproducing stops being news.+STALE_AFTER_SECONDS = 15 * 60++# (provider, kind) -> (monotonic timestamp, client-facing message)+_failures: dict[tuple[str, str], tuple[float, str]] = {}+++def _key(provider: str, kind: CallKind) -> tuple[str, str]:+ return ((provider or "").strip().lower(), kind)+++def record_failure(provider: str, kind: CallKind, message: str) -> None:+ """Remember why a real call to `provider` failed.++ `message` is the text already shown to the caller, so the banner and the+ chat bubble cannot disagree about what went wrong.+ """+ if not provider or not message:+ return+ _failures[_key(provider, kind)] = (time.monotonic(), message)+++def record_success(provider: str, kind: CallKind) -> None:+ """Forget any recorded failure: this provider is serving traffic again."""+ _failures.pop(_key(provider, kind), None)+++def latest_failure(provider: str | None, kind: CallKind) -> str | None:+ """The last real failure for `provider`, if one is still current."""+ if not provider:+ return None+ entry = _failures.get(_key(provider, kind))+ if entry is None:+ return None+ recorded_at, message = entry+ if time.monotonic() - recorded_at > STALE_AFTER_SECONDS:+ _failures.pop(_key(provider, kind), None)+ return None+ return message+++def clear() -> None:+ """Drop every recorded failure. For tests."""+ _failures.clear()diff --git a/tests/unit/api/test_model_providers_endpoint.py b/tests/unit/api/test_model_providers_endpoint.pynew file mode 100644index 000000000..92cc3a504--- /dev/null+++ b/tests/unit/api/test_model_providers_endpoint.py@@ -0,0 +1,120 @@+"""The endpoints that publish which model providers a deployment offers.++The console and any SDK client must be told the same thing, and neither may be+able to read a provider the run mode hides — that is the whole point of moving+the denylist out of the frontend (#2287).+"""++import inspect+import json++import pytest+from fastapi import params as fastapi_params++from api import models as models_api+from api.v1 import llm as v1_llm+from config import model_providers+from session_manager import User+++def _user() -> User:+ return User(user_id="u1", email="u@x", name="U", provider="api_key")+++def _dependency(fn):+ default = inspect.signature(fn).parameters["user"].default+ assert isinstance(default, fastapi_params.Depends)+ return default.dependency++[email protected](autouse=True)+def _fresh_provider_config():+ model_providers.reload()+ yield+ model_providers.reload()+++def _body(response) -> dict:+ assert response.status_code == 200+ return json.loads(response.body)+++def test_console_endpoint_requires_providers_read():+ dep = _dependency(models_api.get_model_providers)+ perms = [cell.cell_contents for cell in dep.__closure__]+ assert "providers:read" in perms+++def test_v1_endpoint_requires_a_catalogue_reading_permission():+ dep = _dependency(v1_llm.model_providers_endpoint)+ required = [cell.cell_contents for cell in dep.__closure__]+ assert any("providers:read" in str(item) for item in required)+++async def test_saas_does_not_publish_ollama(monkeypatch):+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ body = _body(await models_api.get_model_providers(user=_user()))++ assert body["run_mode"] == "saas"+ assert "ollama" not in {entry["name"] for entry in body["providers"]}++[email protected]("mode", ["oss", "on_prem"])+async def test_oss_and_on_prem_still_publish_ollama(monkeypatch, mode):+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)+ body = _body(await models_api.get_model_providers(user=_user()))++ assert body["run_mode"] == mode+ assert "ollama" in {entry["name"] for entry in body["providers"]}+++async def test_the_console_and_v1_endpoints_agree(monkeypatch):+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ console = _body(await models_api.get_model_providers(user=_user()))+ public = _body(await v1_llm.model_providers_endpoint(user=_user()))++ assert console == public+++async def test_an_override_file_drives_the_payload(monkeypatch, tmp_path):+ path = tmp_path / "model_providers.yaml"+ path.write_text(+ "providers:\n - name: ollama\n display_name: Ollama\n modes:\n saas: true\n",+ encoding="utf-8",+ )+ monkeypatch.setenv(model_providers.CONFIG_PATH_ENV, str(path))+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")++ body = _body(await models_api.get_model_providers(user=_user()))+ assert body["providers"] == [{"name": "ollama", "display_name": "Ollama"}]+++async def test_the_provider_list_is_never_stored_by_the_browser():+ """A cached copy outlives the restart that applies a config edit.++ `config/model_providers.yaml` is read once per process, so changing it means+ editing the file and restarting the backend. With a `max-age` on this+ response the browser answers the next page load from its own cache, the+ console redraws the previous provider cards and tabs, and the edit looks+ like it did nothing — for as long as the max-age lasts.+ """+ for response in (+ await models_api.get_model_providers(user=_user()),+ await v1_llm.model_providers_endpoint(user=_user()),+ ):+ assert response.headers["cache-control"] == "no-store"+++async def test_the_catalogue_is_never_stored_by_the_browser():+ """Same trap as the provider list, and the catalogue is what the pickers read."""+ for response in (+ await models_api.get_model_catalog(user=_user()),+ await v1_llm.model_catalog_endpoint(user=_user()),+ ):+ assert response.headers["cache-control"] == "no-store"diff --git a/tests/unit/config/test_model_providers_config.py b/tests/unit/config/test_model_providers_config.pynew file mode 100644index 000000000..6482b97af--- /dev/null+++ b/tests/unit/config/test_model_providers_config.py@@ -0,0 +1,312 @@+"""Per-run-mode model provider visibility (`config/model_providers.yaml`).++The config file is the only place that decides which providers Settings,+Onboarding and the model pickers offer, so these tests pin the rules the file+promises: run mode is the switch, a missing mode key hides, an unlisted+provider is never offered, and an override file wins over the shipped one.+"""++from __future__ import annotations++import pytest++from config import model_providers++[email protected](autouse=True)+def _fresh_config():+ """Each test parses the file itself; never inherit another test's cache."""+ model_providers.reload()+ yield+ model_providers.reload()+++def _write(tmp_path, body: str) -> str:+ path = tmp_path / "model_providers.yaml"+ path.write_text(body, encoding="utf-8")+ return str(path)+++def test_shipped_defaults_keep_ollama_out_of_saas_only(monkeypatch):+ for mode in ("oss", "on_prem"):+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)+ assert "ollama" in model_providers.visible_provider_keys(), mode++ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ assert "ollama" not in model_providers.visible_provider_keys()+++def test_shipped_defaults_expose_the_core_providers_everywhere(monkeypatch):+ for mode in ("oss", "on_prem", "saas"):+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)+ assert {"openai", "anthropic", "watsonx"} <= model_providers.visible_provider_keys(), mode+++def test_both_azure_providers_ship_in_every_mode(monkeypatch):+ """LiteLLM keys Azure twice and the shipped config offers both.++ `azure_ai` is the Foundry catalogue (Llama, Phi, Mistral, Cohere, and the+ gpt-5.x entries Foundry serves); `azure` is Azure OpenAI Service, which is+ the only one of the two carrying the gpt-4.1 family. Dropping either hides+ models the other cannot supply.+ """+ for mode in ("oss", "on_prem", "saas"):+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)+ assert {"azure_ai", "azure"} <= model_providers.visible_provider_keys(), mode+++def test_payload_reports_the_run_mode_it_filtered_on(monkeypatch):+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ payload = model_providers.provider_visibility_payload()++ assert payload["run_mode"] == "saas"+ assert [entry["name"] for entry in payload["providers"]] == [+ entry["name"] for entry in model_providers.visible_providers()+ ]+ assert all(entry["display_name"] for entry in payload["providers"])+++def test_an_unrecognised_run_mode_is_treated_as_oss(monkeypatch):+ monkeypatch.setenv("OPENRAG_RUN_MODE", "not-a-mode")+ assert model_providers.visible_provider_keys() == model_providers.visible_provider_keys("oss")+++def test_a_missing_mode_key_hides_the_provider(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: openai+ display_name: OpenAI+ modes:+ oss: true+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ assert model_providers.visible_provider_keys() == frozenset()++ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert model_providers.visible_provider_keys() == {"openai"}+++def test_a_provider_hidden_in_every_mode_is_omitted_everywhere(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: openai+ modes:+ oss: false+ on_prem: false+ saas: false+""",+ ),+ )+ for mode in ("oss", "on_prem", "saas"):+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)+ assert model_providers.visible_provider_keys() == frozenset(), mode+++def test_flipping_a_mode_in_the_config_needs_no_code_change(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: ollama+ display_name: Ollama+ modes:+ oss: true+ on_prem: true+ saas: true+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ assert "ollama" in model_providers.visible_provider_keys()+++def test_a_provider_the_file_omits_is_never_visible(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: openai+ modes:+ oss: true+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert not model_providers.is_provider_visible("groq")+ assert model_providers.is_provider_visible("OpenAI")+++def test_names_are_normalised_and_duplicates_drop(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: " OpenAI "+ display_name: First+ modes:+ oss: true+ - name: openai+ display_name: Second+ modes:+ oss: true+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert model_providers.visible_provider_entries() == (+ model_providers.ProviderEntry("openai", "First", (), (), ()),+ )+++def test_a_non_boolean_mode_value_hides_rather_than_shows(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: openai+ modes:+ oss: maybe+ - name: anthropic+ modes:+ oss: "true"+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert model_providers.visible_provider_keys() == {"anthropic"}+++def test_display_name_falls_back_to_the_provider_name(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(tmp_path, "providers:\n - name: groq\n modes:\n oss: true\n"),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert model_providers.visible_provider_entries() == (+ model_providers.ProviderEntry("groq", "groq", (), (), ()),+ )+++def test_an_unreadable_override_falls_back_to_the_shipped_file(monkeypatch, tmp_path):+ monkeypatch.setenv(model_providers.CONFIG_PATH_ENV, str(tmp_path / "missing.yaml"))+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ model_providers.reload()+ shipped = model_providers.visible_provider_keys()++ monkeypatch.delenv(model_providers.CONFIG_PATH_ENV)+ model_providers.reload()+ assert shipped == model_providers.visible_provider_keys()+ assert "openai" in shipped+++def test_a_malformed_config_falls_back_rather_than_hiding_everything(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(tmp_path, "providers: not-a-list\n"),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert "openai" in model_providers.visible_provider_keys()+++def test_a_row_can_declare_the_models_its_gateway_serves(monkeypatch, tmp_path):+ """A self-hosted OpenAI-compatible endpoint serves ids LiteLLM cannot know."""+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: openai_like+ display_name: Internal Gateway+ modes:+ oss: true+ models:+ - llama-3.3-70b+ - " qwen2.5-coder-32b "+ - llama-3.3-70b+ embedding_models:+ - bge-m3+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ assert model_providers.visible_provider_entries() == (+ model_providers.ProviderEntry(+ "openai_like",+ "Internal Gateway",+ ("llama-3.3-70b", "qwen2.5-coder-32b"),+ ("bge-m3",),+ (),+ ),+ )+++def test_a_row_that_declares_no_models_reports_empty_tuples(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n models: nope\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert model_providers.visible_provider_entries() == (+ model_providers.ProviderEntry("openai", "openai", (), (), ()),+ )+++def test_a_row_can_exclude_models_by_pattern(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ """+providers:+ - name: openai+ modes:+ oss: true+ exclude_models:+ - GPT-3.5-*+ - gpt-4-0613+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ entry = model_providers.visible_provider_entries()[0]+ # Lowercased on the way in, so a pattern typed in any case still matches.+ assert entry.exclude_models == ("gpt-3.5-*", "gpt-4-0613")+++def test_exclusions_default_to_none_and_ignore_a_non_list(monkeypatch, tmp_path):+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n"+ " exclude_models: gpt-3.5-turbo\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ assert model_providers.visible_provider_entries()[0].exclude_models == ()diff --git a/tests/unit/services/test_llm_gateway.py b/tests/unit/services/test_llm_gateway.pyindex eddd177a9..af247f3e2 100644--- a/tests/unit/services/test_llm_gateway.py+++ b/tests/unit/services/test_llm_gateway.py@@ -1037,3 +1037,216 @@ async def test_stream_sse_reports_a_stream_that_never_yielded_a_chunk():error = json.loads(lines[-2][len("data: ") :])["error"]assert "no content and no tool calls" in error["message"]+++class TestRealFailuresReachTheBanner:+ """What the caller is told and what the banner shows must be the same text.++ The health probe issues its own request and so hits its own error. Recording+ the gateway's actual failure is what lets the banner report the one the+ user's traffic produced.+ """++ @pytest.fixture(autouse=True)+ def _clean(self):+ from services import provider_error_log++ provider_error_log.clear()+ yield+ provider_error_log.clear()++ @pytest.mark.asyncio+ async def test_a_failed_completion_is_recorded_verbatim(self, monkeypatch):+ from services import llm_gateway, provider_error_log++ monkeypatch.setattr(+ llm_gateway, "resolve_call", lambda *a, **k: ("gpt-5.6-luna", "openai", {})+ )++ async def _boom(**_kwargs):+ raise RuntimeError(+ '{"error": {"message": "Function tools with reasoning_effort are not supported"}}'+ )++ monkeypatch.setattr("litellm.acompletion", _boom)++ with pytest.raises(llm_gateway.LlmGatewayError) as raised:+ await llm_gateway.chat_completions({"model": "gpt-5.6-luna", "messages": []})++ recorded = provider_error_log.latest_failure("openai", "chat")+ assert recorded == str(raised.value)+ assert "reasoning_effort" in recorded++ @pytest.mark.asyncio+ async def test_a_later_success_clears_the_record(self, monkeypatch):+ from services import llm_gateway, provider_error_log++ provider_error_log.record_failure("openai", "chat", "stale failure")+ monkeypatch.setattr(llm_gateway, "resolve_call", lambda *a, **k: ("gpt-4o", "openai", {}))++ async def _ok(**_kwargs):+ return {"choices": [{"message": {"content": "hi"}}]}++ monkeypatch.setattr("litellm.acompletion", _ok)++ await llm_gateway.chat_completions({"model": "gpt-4o", "messages": []})++ assert provider_error_log.latest_failure("openai", "chat") is None+++class TestToolsBesideReasoningEffort:+ """gpt-5.x refuses function tools next to its own default reasoning effort.++ OpenRAG never sends `reasoning_effort` — OpenAI applies a non-none default+ to reasoning models and then rejects the pair on /v1/chat/completions,+ naming an explicit "none" as the fix. `drop_params` cannot help: the+ parameter is supported, just not in that combination.+ """++ @pytest.fixture(autouse=True)+ def _clean(self):+ from services import llm_gateway, provider_error_log++ llm_gateway._TOOLS_NEED_REASONING_OFF.clear()+ provider_error_log.clear()+ yield+ llm_gateway._TOOLS_NEED_REASONING_OFF.clear()+ provider_error_log.clear()++ @staticmethod+ def _conflict() -> RuntimeError:+ return RuntimeError(+ "OpenAIException - Function tools with reasoning_effort are not supported "+ "for gpt-5.6-luna in /v1/chat/completions. To use function tools, use "+ "/v1/responses or set reasoning_effort to 'none'."+ )++ def _wire(self, monkeypatch, model="gpt-5.6-luna", supports_none=True):+ from services import llm_gateway++ monkeypatch.setattr(llm_gateway, "resolve_call", lambda *a, **k: (model, "openai", {}))+ monkeypatch.setattr(+ llm_gateway,+ "_model_info",+ lambda _m: {"supports_none_reasoning_effort": supports_none},+ )++ @pytest.mark.asyncio+ async def test_the_call_is_retried_with_reasoning_off(self, monkeypatch):+ from services import llm_gateway++ self._wire(monkeypatch)+ calls = []++ async def _flaky(**kwargs):+ calls.append(kwargs)+ if len(calls) == 1:+ raise self._conflict()+ return {"choices": [{"message": {"content": "hi"}}]}++ monkeypatch.setattr("litellm.acompletion", _flaky)++ await llm_gateway.chat_completions(+ {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}+ )++ assert len(calls) == 2+ assert "reasoning_effort" not in calls[0]+ assert calls[1]["reasoning_effort"] == "none"++ @pytest.mark.asyncio+ async def test_the_next_call_does_not_pay_for_the_lesson_again(self, monkeypatch):+ from services import llm_gateway++ self._wire(monkeypatch)+ calls = []++ async def _flaky(**kwargs):+ calls.append(kwargs)+ if len(calls) == 1:+ raise self._conflict()+ return {"choices": [{"message": {"content": "hi"}}]}++ monkeypatch.setattr("litellm.acompletion", _flaky)+ body = {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}++ await llm_gateway.chat_completions(dict(body))+ await llm_gateway.chat_completions(dict(body))++ assert len(calls) == 3+ assert calls[2]["reasoning_effort"] == "none"++ @pytest.mark.asyncio+ async def test_an_effort_the_caller_chose_is_never_overridden(self, monkeypatch):+ """Silently disabling reasoning someone asked for is worse than the error."""+ from services import llm_gateway++ self._wire(monkeypatch)+ monkeypatch.setattr(+ llm_gateway,+ "_LITELLM_FORWARDED_PARAMS",+ (*llm_gateway._LITELLM_FORWARDED_PARAMS, "reasoning_effort"),+ )+ calls = []++ async def _always_fails(**kwargs):+ calls.append(kwargs)+ raise self._conflict()++ monkeypatch.setattr("litellm.acompletion", _always_fails)++ with pytest.raises(llm_gateway.LlmGatewayError):+ await llm_gateway.chat_completions(+ {+ "model": "gpt-5.6-luna",+ "messages": [],+ "tools": [{"type": "function"}],+ "reasoning_effort": "high",+ }+ )++ assert len(calls) == 1+ assert calls[0]["reasoning_effort"] == "high"++ @pytest.mark.asyncio+ async def test_a_model_that_cannot_take_none_reports_the_error(self, monkeypatch):+ """Nothing to retry with — only the Responses API can serve tools there."""+ from services import llm_gateway++ self._wire(monkeypatch, supports_none=False)+ calls = []++ async def _always_fails(**kwargs):+ calls.append(kwargs)+ raise self._conflict()++ monkeypatch.setattr("litellm.acompletion", _always_fails)++ with pytest.raises(llm_gateway.LlmGatewayError):+ await llm_gateway.chat_completions(+ {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}+ )++ # Raised on the first attempt: there is no second value to try.+ assert len(calls) == 1+ assert "reasoning_effort" not in calls[0]++ @pytest.mark.asyncio+ async def test_an_unrelated_failure_is_not_retried(self, monkeypatch):+ from services import llm_gateway++ self._wire(monkeypatch)+ calls = []++ async def _no_credits(**kwargs):+ calls.append(kwargs)+ raise RuntimeError("You have no credits remaining.")++ monkeypatch.setattr("litellm.acompletion", _no_credits)++ with pytest.raises(llm_gateway.LlmGatewayError):+ await llm_gateway.chat_completions(+ {"model": "gpt-5.6-luna", "messages": [], "tools": [{"type": "function"}]}+ )++ assert len(calls) == 1diff --git a/tests/unit/services/test_model_catalog.py b/tests/unit/services/test_model_catalog.pyindex 18d08c6ed..b3e5c4203 100644--- a/tests/unit/services/test_model_catalog.py+++ b/tests/unit/services/test_model_catalog.py@@ -9,14 +9,25 @@ the same payload.from __future__ import annotations+import pytest++from config import model_providersfrom services import model_catalog[email protected](autouse=True)+def _fresh_provider_config():+ """Run-mode tests below rewrite the provider config; don't leak the cache."""+ model_providers.reload()+ yield+ model_providers.reload()++def test_catalog_only_groups_supported_providers_with_their_models_and_forms() -> None:providers = {entry["key"]: entry for entry in model_catalog.catalog()["providers"]}- assert set(providers) == model_catalog.SUPPORTED_PROVIDER_KEYS- for key in model_catalog.SUPPORTED_PROVIDER_KEYS:+ assert set(providers) == model_catalog.supported_provider_keys()+ for key in model_catalog.supported_provider_keys():assert key in providers, keyassert providers[key]["credential_fields"], key@@ -85,6 +96,29 @@ def test_is_known_provider_accepts_litellm_handlers() -> None:assert not model_catalog.is_known_provider("")+def test_is_known_provider_covers_every_provider_litellm_can_route() -> None:+ """`provider_list` is enum members, not strings.++ Reading them with str() gives "LlmProviders.OPENAI", so the check silently+ fell through to the credential-form specs — which omit ~26 routable+ providers. Their `provider/model` ids were then left unsplit and billed to+ whatever the default provider was.+ """+ import litellm++ routable = {getattr(value, "value", str(value)) for value in litellm.provider_list}+ assert routable, "litellm exposes no provider list"+ missing = sorted(key for key in routable if not model_catalog.is_known_provider(key))+ assert not missing, missing+++def test_a_routable_prefix_without_a_credential_form_still_splits() -> None:+ from services.llm_gateway import split_model_id++ assert split_model_id("zai/glm-4.6") == ("zai", "glm-4.6")+ assert split_model_id("not-a-provider/x") == (None, "not-a-provider/x")++def test_exported_model_ids_route_back_to_their_owner() -> None:"""A `/v1/models` id must resolve to the provider that owns it.@@ -105,3 +139,291 @@ def test_exported_model_ids_route_back_to_their_owner() -> None:openai_rows = [row for row in payload["data"] if row["owned_by"] == "openai"]assert openai_rowsassert all("/" not in row["id"] for row in openai_rows)+++def test_catalog_hides_a_provider_the_run_mode_does_not_expose(monkeypatch) -> None:+ """SaaS must not advertise Ollama; oss and on-prem must keep it."""+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ saas = {entry["key"] for entry in model_catalog.catalog()["providers"]}+ assert "ollama" not in saas+ assert "openai" in saas++ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert "ollama" in {entry["key"] for entry in model_catalog.catalog()["providers"]}+++def test_catalog_publishes_azure_ai_where_the_config_enables_it(monkeypatch) -> None:+ for mode in ("oss", "on_prem", "saas"):+ monkeypatch.setenv("OPENRAG_RUN_MODE", mode)+ providers = {entry["key"]: entry for entry in model_catalog.catalog()["providers"]}+ assert "azure_ai" in providers, mode+ azure = providers["azure_ai"]+ assert azure["name"] == "Azure AI Foundry"+ assert azure["models"], mode+ assert azure["embedding_models"], mode+ assert {"api_base", "api_key"} <= {field["key"] for field in azure["credential_fields"]}+++def test_azure_openai_is_the_row_that_carries_the_gpt_41_family(monkeypatch) -> None:+ """The two Azure rows are not interchangeable.++ LiteLLM files Azure OpenAI Service under `azure` and the Foundry catalogue+ under `azure_ai`, and only the former lists gpt-4.1. Offering just+ `azure_ai` — as the config first did — leaves no way to pick gpt-4.1 from+ the dropdown at all.+ """+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ providers = {entry["key"]: entry for entry in model_catalog.catalog()["providers"]}++ assert providers["azure"]["name"] == "Azure OpenAI"+ assert "gpt-4.1" in {entry["model"] for entry in providers["azure"]["models"]}+ assert "gpt-4.1" not in {entry["model"] for entry in providers["azure_ai"]["models"]}+ # The prefix is stripped for display but re-attached on the way out, or the+ # id would be called with the default provider's credentials.+ assert "azure:gpt-4.1" in {row["id"] for row in model_catalog.openai_models_list()["data"]}+++def test_openai_models_list_drops_a_hidden_providers_models(monkeypatch) -> None:+ monkeypatch.setenv("OPENRAG_RUN_MODE", "saas")+ owners = {row["owned_by"] for row in model_catalog.openai_models_list()["data"]}+ assert "ollama" not in owners++ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert "ollama" in {row["owned_by"] for row in model_catalog.openai_models_list()["data"]}+++def test_the_config_display_name_names_the_provider(monkeypatch, tmp_path) -> None:+ """A catalogue entry is labelled by the config file, not by LiteLLM."""+ path = tmp_path / "model_providers.yaml"+ path.write_text(+ "providers:\n - name: openai\n display_name: House Gateway\n"+ " modes:\n oss: true\n",+ encoding="utf-8",+ )+ monkeypatch.setenv(model_providers.CONFIG_PATH_ENV, str(path))+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ providers = model_catalog.catalog()["providers"]+ assert [entry["key"] for entry in providers] == ["openai"]+ assert providers[0]["name"] == "House Gateway"+++def _write_providers(tmp_path, body: str) -> str:+ path = tmp_path / "model_providers.yaml"+ path.write_text(body, encoding="utf-8")+ return str(path)+++def test_a_custom_gateway_can_declare_the_models_it_serves(monkeypatch, tmp_path) -> None:+ """LiteLLM's table has no ids for a self-hosted OpenAI-compatible endpoint.++ Without config-declared ids the provider would render a card and a working+ credential form, then offer an empty model picker.+ """+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ """+providers:+ - name: openai_like+ display_name: Internal LLM Gateway+ modes:+ oss: true+ models:+ - llama-3.3-70b-instruct+ embedding_models:+ - bge-m3+""",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ providers = model_catalog.catalog()["providers"]+ assert [entry["key"] for entry in providers] == ["openai_like"]+ gateway = providers[0]+ assert gateway["name"] == "Internal LLM Gateway"+ assert [entry["model"] for entry in gateway["models"]] == ["llama-3.3-70b-instruct"]+ assert [entry["model"] for entry in gateway["embedding_models"]] == ["bge-m3"]+ # The endpoint has to be enterable, or the gateway is unreachable.+ assert "api_base" in {field["key"] for field in gateway["credential_fields"]}+++def test_a_declared_gateway_model_routes_back_to_its_own_provider(monkeypatch, tmp_path) -> None:+ from services.llm_gateway import PROVIDER_SEPARATOR, split_model_id++ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: openai_like\n modes:\n oss: true\n"+ " models:\n - llama-3.3-70b-instruct\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ ids = [row["id"] for row in model_catalog.openai_models_list()["data"]]+ assert ids == [f"openai_like{PROVIDER_SEPARATOR}llama-3.3-70b-instruct"]+ assert split_model_id(ids[0]) == ("openai_like", "llama-3.3-70b-instruct")+++def test_declared_ids_do_not_duplicate_what_litellm_already_lists(monkeypatch, tmp_path) -> None:+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n"+ " models:\n - gpt-4o\n - house-tuned-gpt\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ models = [entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]]+ assert models.count("gpt-4o") == 1+ assert "house-tuned-gpt" in models+++def test_a_configured_exclusion_keeps_models_out_of_the_catalogue(monkeypatch, tmp_path) -> None:+ """`exclude_models` is how a deployment refuses a model outright.++ Distinct from the two automatic paths: a retired model is dropped without+ being named here, and one retiring soon is only hidden behind a toggle in+ the picker.+ """+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n"+ " exclude_models:\n - gpt-3.5-*\n - gpt-4o\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ models = {entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]}+ assert not any(name.startswith("gpt-3.5") for name in models), sorted(models)[:5]+ assert "gpt-4o" not in models+ # Only what was named: a wildcard must not take the neighbours with it.+ assert "gpt-4o-mini" in models+ assert any(name.startswith("gpt-4.1") for name in models)+++def test_an_exclusion_also_overrides_a_declared_model(monkeypatch, tmp_path) -> None:+ """Two rows that contradict each other resolve one way, predictably."""+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n"+ " models:\n - house-tuned-gpt\n"+ " exclude_models:\n - house-tuned-*\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ models = {entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]}+ assert "house-tuned-gpt" not in models+++def test_an_excluded_model_is_gone_from_v1_models_too(monkeypatch, tmp_path) -> None:+ """The picker and the OpenAI-compatible list must not disagree."""+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n"+ " exclude_models:\n - gpt-4o\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ assert "gpt-4o" not in {row["id"] for row in model_catalog.openai_models_list()["data"]}+++def test_exclusions_reach_the_live_provider_lists_too(monkeypatch, tmp_path) -> None:+ """Ollama, watsonx, OpenAI and Anthropic are also listed by asking the provider.++ That path never touches the catalogue, so without filtering it an excluded+ id vanishes from Settings and walks straight back in through onboarding.+ """+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: openai\n modes:\n oss: true\n"+ " exclude_models:\n - gpt-3.5-*\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ payload = {+ "language_models": [+ {"value": "gpt-3.5-turbo", "label": "gpt-3.5-turbo"},+ {"value": "gpt-4o", "label": "gpt-4o"},+ ],+ "embedding_models": [{"value": "text-embedding-3-small"}],+ }+ filtered = model_catalog.hide_excluded_live_models("openai", payload)++ assert [entry["value"] for entry in filtered["language_models"]] == ["gpt-4o"]+ assert [entry["value"] for entry in filtered["embedding_models"]] == ["text-embedding-3-small"]+ # The caller's payload is not mutated; only the response is filtered.+ assert len(payload["language_models"]) == 2+++def test_a_provider_with_no_exclusions_is_passed_through_untouched(monkeypatch) -> None:+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ payload = {"language_models": [{"value": "llama3"}]}+ assert model_catalog.hide_excluded_live_models("ollama", payload) is payload+++def test_a_live_payload_that_is_not_a_dict_survives(monkeypatch) -> None:+ """The live endpoints can return an error body; filtering must not raise."""+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ assert model_catalog.hide_excluded_live_models("openai", None) is None+ assert model_catalog.hide_excluded_live_models("openai", []) == []+++def test_the_shipped_config_keeps_the_legacy_openai_generation_out(monkeypatch) -> None:+ """The exclusions the shipped file actually carries."""+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ models = {+ entry["model"]+ for provider in model_catalog.catalog()["providers"]+ if provider["key"] == "openai"+ for entry in provider["models"]+ }++ assert not any(name.startswith("gpt-3.5") for name in models)+ assert "gpt-5.6-luna" not in models+ # Everything else the generation filter must not have taken with it.+ assert "gpt-4o" in models+ assert any(name.startswith("gpt-5.6") for name in models)+++def test_a_plain_name_also_excludes_its_regional_listings(monkeypatch, tmp_path) -> None:+ """Azure lists the same model per region; suppressing it means all of them."""+ monkeypatch.setenv(+ model_providers.CONFIG_PATH_ENV,+ _write_providers(+ tmp_path,+ "providers:\n - name: azure\n modes:\n oss: true\n"+ " exclude_models:\n - gpt-5.6-luna\n",+ ),+ )+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")++ models = {entry["model"] for entry in model_catalog.catalog()["providers"][0]["models"]}+ assert not any(name.endswith("gpt-5.6-luna") for name in models), sorted(+ name for name in models if "luna" in name+ )+ # A region's other models are untouched.+ assert any(name.startswith("eu/") for name in models)+++def test_the_shipped_config_excludes_on_azure_as_well_as_openai(monkeypatch) -> None:+ """The same model is served by two rows, so it takes two exclusions."""+ monkeypatch.setenv("OPENRAG_RUN_MODE", "oss")+ ids = {row["id"] for row in model_catalog.openai_models_list()["data"]}++ assert not [model_id for model_id in ids if "gpt-3.5" in model_id or "luna" in model_id]diff --git a/tests/unit/services/test_provider_error_log.py b/tests/unit/services/test_provider_error_log.pynew file mode 100644index 000000000..057967e55--- /dev/null+++ b/tests/unit/services/test_provider_error_log.py@@ -0,0 +1,72 @@+"""The gateway's record of what real traffic hit.++The health banner used to show only what its own probe hit, and a probe cannot+reproduce the request that failed — OpenAI checks request shape before billing,+so a probe reports "no credits remaining" while the agent's own call reports a+400 about its parameters. These pin that the record follows real calls and+does not outlive them.+"""++from __future__ import annotations++import pytest++from services import provider_error_log++[email protected](autouse=True)+def _clean():+ provider_error_log.clear()+ yield+ provider_error_log.clear()+++def test_a_failure_is_readable_until_the_next_success():+ provider_error_log.record_failure("openai", "chat", "gpt-5.6-luna: bad params")+ assert provider_error_log.latest_failure("openai", "chat") == "gpt-5.6-luna: bad params"++ provider_error_log.record_success("openai", "chat")+ assert provider_error_log.latest_failure("openai", "chat") is None+++def test_chat_and_embedding_are_recorded_apart():+ """One kind recovering must not clear the other's still-live failure."""+ provider_error_log.record_failure("openai", "chat", "chat is broken")+ provider_error_log.record_failure("openai", "embedding", "embedding is broken")++ provider_error_log.record_success("openai", "chat")++ assert provider_error_log.latest_failure("openai", "chat") is None+ assert provider_error_log.latest_failure("openai", "embedding") == "embedding is broken"+++def test_providers_do_not_share_a_record():+ provider_error_log.record_failure("openai", "chat", "openai is broken")+ assert provider_error_log.latest_failure("anthropic", "chat") is None+++def test_the_provider_name_is_matched_loosely():+ provider_error_log.record_failure(" OpenAI ", "chat", "boom")+ assert provider_error_log.latest_failure("openai", "chat") == "boom"+++def test_a_failure_nobody_reproduces_stops_speaking(monkeypatch):+ """A provider that fails and is then never called again must not pin the banner."""+ provider_error_log.record_failure("openai", "chat", "boom")++ clock = [0.0]+ monkeypatch.setattr(provider_error_log.time, "monotonic", lambda: clock[0])+ provider_error_log.record_failure("openai", "chat", "boom")++ clock[0] = provider_error_log.STALE_AFTER_SECONDS - 1+ assert provider_error_log.latest_failure("openai", "chat") == "boom"++ clock[0] = provider_error_log.STALE_AFTER_SECONDS + 1+ assert provider_error_log.latest_failure("openai", "chat") is None+++def test_nothing_is_recorded_without_a_provider_or_a_message():+ provider_error_log.record_failure("", "chat", "boom")+ provider_error_log.record_failure("openai", "chat", "")+ assert provider_error_log.latest_failure("openai", "chat") is None+ assert provider_error_log.latest_failure(None, "chat") is Nonediff --git a/tests/unit/test_models_api_errors.py b/tests/unit/test_models_api_errors.pyindex 15924fa3d..4bbb2a2b0 100644--- a/tests/unit/test_models_api_errors.py+++ b/tests/unit/test_models_api_errors.py@@ -138,17 +138,26 @@ async def test_get_ibm_models_returns_project_configuration_error():@pytest.mark.asyncioasync def test_get_model_catalog_returns_only_supported_providers():+ """The catalogue publishes exactly what this run mode's config exposes.++ Not a fixed four: `config/model_providers.yaml` decides, so pinning a+ literal set here would only re-hardcode the list the config replaced.+ """+ from services.model_catalog import supported_provider_keys+response = await models_api.get_model_catalog(user=SimpleNamespace())assert isinstance(response, JSONResponse)assert response.status_code == 200payload = json.loads(response.body)keys = {provider["key"] for provider in payload["providers"]}- assert keys == {"openai", "anthropic", "ollama", "watsonx"}+ assert keys == supported_provider_keys()openai = next(p for p in payload["providers"] if p["key"] == "openai")assert openai["models"]assert openai["embedding_models"]cache_control = response.headers.get("cache-control") or response.headers.get("Cache-Control")- assert cache_control == "private, max-age=3600"+ # Never `max-age`: a cached copy outlives the restart that applies a+ # config edit, so the picker keeps showing the previous model list.+ assert cache_control == "no-store"@pytest.mark.asynciodiff --git a/tests/unit/test_settings_provider_removal_defaults.py b/tests/unit/test_settings_provider_removal_defaults.pyindex 7bd4f36cb..fad77f4ff 100644--- a/tests/unit/test_settings_provider_removal_defaults.py+++ b/tests/unit/test_settings_provider_removal_defaults.py@@ -3,8 +3,16 @@When a provider is removed and it was the active LLM or embedding provider,the backend should fall back to another configured provider AND select asensible default model (not an empty string).++Ollama is the fallback these cases exercise, and a fallback helper will not+hand back a provider the run mode hides. The module pins a run mode that+offers Ollama so these stay about fallback selection whatever+`config/model_providers.yaml` says; `test_model_providers_config` covers the+hiding itself."""+import pytest+from api.settings.helpers import (_default_embedding_model,_default_llm_model,@@ -32,6 +40,11 @@ from config.model_constants import (# ---------------------------------------------------------------------------[email protected](autouse=True)+def _run_mode_that_offers_ollama(monkeypatch):+ monkeypatch.setenv("OPENRAG_RUN_MODE", "on_prem")++def _make_config(*,openai=False,