OpenRAG is a comprehensive, single package Retrieval-Augmented Generation platform built on Langflow, Docling, and Opensearch.
Python
# Which model providers OpenRAG offers, per run mode.
#
# This file is the single source of truth for provider visibility in the
# Settings provider cards, the Onboarding tabs, and the model catalogue that
# feeds every agent/ingest picker. The backend filters on it; the UI renders
# whatever the API returns, so there is no second denylist in the frontend.
#
# Keys under `modes` match OPENRAG_RUN_MODE: oss | on_prem | saas.
# true -> the provider is shown in that mode
# false -> the provider is hidden
# A missing mode key means hidden, so adding a new run mode can never expose a
# provider by accident.
#
# A provider that is not listed here is not offered anywhere in the UI. A model
# id for an unlisted provider can still be typed by hand and will be routed if
# the gateway already supports it.
#
# Set OPENRAG_MODEL_PROVIDERS_CONFIG to another YAML file to override this list
# for a single deployment; this file is the default.
#
# A row may also declare `models` / `embedding_models`. LiteLLM's bundled table
# already knows the public vendors, so those rows need nothing. A self-hosted
# OpenAI-compatible gateway serves whatever its operator deployed, and those ids
# can only come from here — see the commented example at the end of this file.
#
# `exclude_models` keeps ids out of the pickers. `*` and `?` wildcards work, and
# matching is against the id as the picker shows it (`gpt-3.5-turbo`, not
# `openai/gpt-3.5-turbo`). Exclusion is applied last, so it also overrides a
# `models` row on the same provider. Models the provider has already retired are
# dropped without being listed here, and ones retiring within 90 days are hidden
# behind a "Show N retiring" row in the picker — use this for the ones a
# deployment does not want offered at all.
providers:
- name: openai
display_name: OpenAI
modes:
oss: true
on_prem: true
saas: true
exclude_models:
- gpt-3.5-* # a whole generation, superseded and near retirement
# Refuses function tools on /v1/chat/completions. Both generations, and
# the trailing `*` for the dated snapshots (`gpt-5.6-luna-2026-07-09`):
# an exact pattern matches the undated id only, so a snapshot released
# later reappears in the picker on its own.
#
# LiteLLM's table reports `supports_function_calling: true` for every
# luna row, including the 5.6 that does not — which is why the exclusion
# is hand-maintained here rather than read off the catalogue. Treat a new
# luna generation as defective until someone proves otherwise with a real
# tool call: an id excluded here can still be typed by hand, so hiding a
# working model costs less than offering a broken one.
- gpt-5.6-luna*
- gpt-6-luna*
- name: ollama
display_name: Ollama
modes:
oss: true
on_prem: false
saas: false # local inference is not offered in SaaS
- name: watsonx
display_name: IBM watsonx.ai
modes:
oss: true
on_prem: true
saas: true
exclude_models:
# LiteLLM ids that do not match the API ids in IBM's model documentation.
- bigscience/mt0-xxl-13b
- google/flan-t5-xl-3b
- ibm/granite-guardian-3-2-2b
- ibm/granite-guardian-3-3-8b
- meta-llama/llama-4-maverick-17b
- mistralai/mistral-small-2503
- mistralai/pixtral-12b-2409
# Models reported unusable against watsonx.ai in tracker issue 82237.
- mistralai/mistral-medium-2505 # fails during streamed chat responses
- ibm/granite-3-3-8b-instruct # unsupported or removed by watsonx.ai
- ibm/granite-3-8b-instruct
- meta-llama/llama-3-2-90b-vision-instruct
- meta-llama/llama-3-2-11b-vision-instruct
- meta-llama/llama-3-2-3b-instruct
- meta-llama/llama-3-2-1b-instruct
- mistralai/mistral-large
- meta-llama/llama-guard-3-11b-vision
- ibm/granite-vision-3-2-2b
- ibm/granite-ttm-1536-96-r2
- ibm/granite-ttm-1024-96-r2
- ibm/granite-ttm-512-96-r2
- core42/jais-13b-chat
- ibm/granite-13b-chat-v2
- ibm/granite-13b-instruct-v2
- sdaia/allam-1-13b-instruct
- bigscience/mt0-xxl
# Accept images on watsonx.ai (`image_chat` in its foundation_model_specs)
# but LiteLLM does not flag them `supports_vision`. Every watsonx id it does
# flag is withdrawn and excluded above, so without these the VLM picker
# offers no watsonx model at all.
vision_models:
- meta-llama/llama-4-maverick-17b-128e-instruct-fp8
- mistralai/mistral-small-3-1-24b-instruct-2503
# watsonx.ai running on a Cloud Pak for Data / IBM Software Hub cluster. It
# is a second front door onto the same `/ml/v1` API as the row above, kept
# separate because the two need their own credentials: the cluster
# authenticates with a ZenApiKey built from a CPD username and API key, not
# with an IBM Cloud IAM key, and it is reached at a cluster URL rather than a
# region endpoint. OpenRAG routes it through LiteLLM's `watsonx` provider —
# see `enhancements/providers/watsonx/onprem.py`.
#
# The two lists below are a *fallback*, not the catalogue. OpenRAG asks the
# cluster itself what it serves (`GET /ml/v1/foundation_model_specs`) and
# shows that in the pickers, so a model the cluster does not have is never
# offered. These rows are what it falls back to while the cluster is
# unreachable or before any credentials are entered — keep them plausible for
# your fleet, but there is no need to keep them exact.
- name: watsonx_onprem
display_name: IBM watsonx.ai
badge: On-prem
modes:
oss: true
on_prem: true
saas: false # a cluster-hosted deployment has no place in multi-tenant SaaS
# Checked against a Cloud Pak for Data 5.x cluster, which serves exactly
# these two. Note the embedding id has no `-v2` suffix: `slate-30m-
# english-rtrvr-v2` is an IBM Cloud id and 404s here, which is the kind of
# near-miss that makes a guessed fallback worse than none.
models:
- openai/gpt-oss-120b
embedding_models:
- ibm/slate-30m-english-rtrvr
# Red Hat OpenShift AI: models served by KServe + vLLM inside the customer's
# own OpenShift cluster. vLLM speaks the OpenAI API, so OpenRAG routes this
# through LiteLLM's `hosted_vllm` provider — see
# `enhancements/providers/redhat/openshift_ai.py`, which also explains why it
# is not `openai_like` (that key silently drops the per-provider TLS setting a
# cluster CA needs).
#
# On in on_prem only as shipped: the cluster is the customer's own, and its two
# endpoint URLs and token are entered in Settings. Elsewhere it is enabled per
# deployment with OPENRAG_MODEL_PROVIDERS_CONFIG rather than offered to everyone.
#
# No `models:` / `embedding_models:` rows are shipped. Model ids are the
# InferenceServices' `--served-model-name` values, which only the deployment
# knows, so OpenRAG asks each endpoint what it serves (`GET /v1/models`):
# with the saved credentials for the catalogue, and with the typed ones
# (`POST /models/rhoai/discover`) during onboarding. A model the cluster does
# not serve is never offered; while it cannot be reached, the pickers accept
# a typed model id.
#
# A deployment's own config may still add rows as a fallback for while the
# endpoints are unreachable, e.g. `models: [gpt-oss-120b]`. Use the
# served-model-name, not the image tag.
- name: rhoai
display_name: Red Hat OpenShift AI
modes:
oss: false
on_prem: true
saas: false
- name: anthropic
display_name: Anthropic
modes:
oss: true
on_prem: true
saas: true
- name: azure_ai
display_name: Azure AI Foundry
modes:
oss: false
on_prem: true
saas: true
# Azure OpenAI Service, which LiteLLM keys separately from Foundry above:
# `azure_ai` is the Foundry catalogue (Llama, Phi, Mistral, Cohere, and the
# gpt-5.x entries Foundry serves), `azure` is the OpenAI Service deployment
# where the gpt-4.1 family lives. Both can be offered at once.
- name: azure
display_name: Azure OpenAI
modes:
oss: true
on_prem: true
saas: true
# Azure serves the same OpenAI models, so it needs the same exclusions.
# The plain name covers the regional ids (`eu/`, `us/`, `global/`) too;
# see the openai row above for why both luna generations are wildcarded.
exclude_models:
- gpt-3.5-*
- gpt-5.6-luna*
- gpt-6-luna*
# Azure lets you name a deployment anything, so LiteLLM's table cannot know
# yours. Declare them here and they appear in the pickers for everyone;
# otherwise each user has to type the name into the model box by hand.
# models:
# - prod-gpt41-eastus
# embedding_models:
# - prod-embed-3-large
# A self-hosted OpenAI-compatible endpoint (vLLM, LM Studio, an internal
# gateway). Use one of LiteLLM's passthrough provider keys — `openai_like`,
# `hosted_vllm`, `lm_studio`, `llamafile`, `custom_openai` — rather than
# inventing a name: the key has to be one the gateway can route, or the
# `provider:model` ids it publishes resolve to the default provider and get
# called with the wrong credentials. (A name of OpenRAG's own works only if
# it is registered in `PROVIDER_ROUTE_ALIASES` in `services/model_catalog.py`
# alongside the LiteLLM key it routes as, the way `watsonx_onprem` is.) The
# endpoint and key are entered in Settings; only the model ids belong here.
#
# - name: openai_like
# display_name: Internal LLM Gateway
# modes:
# oss: true
# on_prem: true
# saas: false
# models:
# - llama-3.3-70b-instruct
# - qwen2.5-coder-32b
# embedding_models:
# - bge-m3