Skip to content

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

feat(llms): add deepinfra as an openai-compatible provider (#7459)

Georgi Atsevd38ceafb68c6
diff --git a/docs/edge/ar/concepts/llms.mdx b/docs/edge/ar/concepts/llms.mdx
index 863a80ee3..6da174f9e 100644
--- a/docs/edge/ar/concepts/llms.mdx
+++ b/docs/edge/ar/concepts/llms.mdx
@@ -1012,6 +1012,39 @@ mode: "wide"
uv add 'crewai[litellm]'
```
</Accordion>
+
+ <Accordion title="DeepInfra">
+ يتصل CrewAI بـ DeepInfra عبر عميل OpenAI الأصلي، لذا لا حاجة لتثبيت LiteLLM.
+
+ عيّن متغيرات البيئة التالية في ملف `.env`:
+ ```toml Code
+ # Required
+ DEEPINFRA_API_KEY=<your-api-key>
+
+ # Optional, point at a proxy or a private deployment endpoint
+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
+ ```
+
+ مثال الاستخدام في مشروع CrewAI:
+ ```python Code
+ llm = LLM(
+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
+ temperature=0.7
+ )
+ ```
+
+ <Info>
+ ميزات DeepInfra:
+ - نماذج مفتوحة الأوزان (Llama وQwen وDeepSeek وGLM وMistral وغيرها) خلف API واحد متوافق مع OpenAI
+ - استدعاء الأدوات والبث والمخرجات المنظمة بمخطط JSON
+ - تسعير لكل token وطول السياق منشوران في كتالوج النماذج
+ </Info>
+
+ معرّفات النماذج بصيغة `org/model` أصلًا، لذا يتكوّن المرجع الكامل من ثلاثة أجزاء،
+ مثل `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. راجع
+ [كتالوج النماذج المنسّق لـ CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)،
+ وهو عام ولا يتطلب مصادقة، للحصول على المعرّفات الحالية وأطوال السياق والأسعار.
+ </Accordion>
</AccordionGroup>
## بث الاستجابات
diff --git a/docs/edge/en/concepts/llms.mdx b/docs/edge/en/concepts/llms.mdx
index 957a52e94..084c60082 100644
--- a/docs/edge/en/concepts/llms.mdx
+++ b/docs/edge/en/concepts/llms.mdx
@@ -1155,6 +1155,39 @@ In this section, you'll find detailed examples that help you select, configure,
uv add 'crewai[litellm]'
```
</Accordion>
+
+ <Accordion title="DeepInfra">
+ CrewAI talks to DeepInfra through its native OpenAI client, so no LiteLLM install is needed.
+
+ Set the following environment variables in your `.env` file:
+ ```toml Code
+ # Required
+ DEEPINFRA_API_KEY=<your-api-key>
+
+ # Optional, point at a proxy or a private deployment endpoint
+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
+ ```
+
+ Example usage in your CrewAI project:
+ ```python Code
+ llm = LLM(
+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
+ temperature=0.7
+ )
+ ```
+
+ <Info>
+ DeepInfra features:
+ - Open-weight models (Llama, Qwen, DeepSeek, GLM, Mistral and more) behind one OpenAI-compatible API
+ - Tool calling, streaming and JSON schema structured outputs
+ - Per-token pricing and context length published in the model catalog
+ </Info>
+
+ Model ids are already in `org/model` form, so a full reference carries three segments,
+ for example `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. See the
+ [model catalog curated for CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
+ which is public and needs no authentication, for current ids, context lengths and pricing.
+ </Accordion>
</AccordionGroup>
## Streaming Responses
diff --git a/docs/edge/ko/concepts/llms.mdx b/docs/edge/ko/concepts/llms.mdx
index 63b3f861c..df477da70 100644
--- a/docs/edge/ko/concepts/llms.mdx
+++ b/docs/edge/ko/concepts/llms.mdx
@@ -755,6 +755,39 @@ CrewAI는 고유한 기능, 인증 방법, 모델 역량을 제공하는 다양
uv add 'crewai[litellm]'
```
</Accordion>
+
+ <Accordion title="DeepInfra">
+ CrewAI는 네이티브 OpenAI 클라이언트로 DeepInfra에 연결하므로 LiteLLM 설치가 필요하지 않습니다.
+
+ `.env` 파일에 다음 환경 변수를 설정하십시오:
+ ```toml Code
+ # Required
+ DEEPINFRA_API_KEY=<your-api-key>
+
+ # Optional, point at a proxy or a private deployment endpoint
+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
+ ```
+
+ CrewAI 프로젝트에서의 예시 사용법:
+ ```python Code
+ llm = LLM(
+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
+ temperature=0.7
+ )
+ ```
+
+ <Info>
+ DeepInfra 특징:
+ - 하나의 OpenAI 호환 API로 제공되는 오픈 웨이트 모델 (Llama, Qwen, DeepSeek, GLM, Mistral 등)
+ - 도구 호출, 스트리밍, JSON 스키마 구조화 출력 지원
+ - 모델 카탈로그에 공개된 토큰 단위 가격과 컨텍스트 길이
+ </Info>
+
+ 모델 ID는 이미 `org/model` 형식이므로 전체 참조는 세 개의 세그먼트로 구성됩니다.
+ 예: `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. 현재 ID, 컨텍스트 길이, 가격은
+ 인증 없이 공개된 [CrewAI용으로 선별된 모델 카탈로그](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)를
+ 참조하세요.
+ </Accordion>
</AccordionGroup>
## 스트리밍 응답
diff --git a/docs/edge/pt-BR/concepts/llms.mdx b/docs/edge/pt-BR/concepts/llms.mdx
index 45ef20bbc..ce46c01dd 100644
--- a/docs/edge/pt-BR/concepts/llms.mdx
+++ b/docs/edge/pt-BR/concepts/llms.mdx
@@ -728,6 +728,39 @@ Nesta seção, você encontrará exemplos detalhados que ajudam a selecionar, co
uv add 'crewai[litellm]'
```
</Accordion>
+
+ <Accordion title="DeepInfra">
+ O CrewAI se comunica com o DeepInfra pelo cliente OpenAI nativo, então não é necessário instalar o LiteLLM.
+
+ Defina as seguintes variáveis de ambiente no seu arquivo `.env`:
+ ```toml Code
+ # Required
+ DEEPINFRA_API_KEY=<your-api-key>
+
+ # Optional, point at a proxy or a private deployment endpoint
+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai
+ ```
+
+ Exemplo de uso em seu projeto CrewAI:
+ ```python Code
+ llm = LLM(
+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
+ temperature=0.7
+ )
+ ```
+
+ <Info>
+ Recursos do DeepInfra:
+ - Modelos de pesos abertos (Llama, Qwen, DeepSeek, GLM, Mistral e outros) por trás de uma única API compatível com OpenAI
+ - Chamada de ferramentas, streaming e saídas estruturadas com JSON schema
+ - Preço por token e tamanho de contexto publicados no catálogo de modelos
+ </Info>
+
+ Os IDs dos modelos já estão no formato `org/model`, então uma referência completa tem três segmentos,
+ por exemplo `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. Consulte o
+ [catálogo de modelos selecionado para o CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),
+ que é público e não exige autenticação, para obter os IDs atuais, tamanhos de contexto e preços.
+ </Accordion>
</AccordionGroup>
## Respostas em streaming
diff --git a/lib/crewai/src/crewai/llm.py b/lib/crewai/src/crewai/llm.py
index 2ba1e6409..fff8f7e27 100644
--- a/lib/crewai/src/crewai/llm.py
+++ b/lib/crewai/src/crewai/llm.py
@@ -197,6 +197,7 @@ SUPPORTED_NATIVE_PROVIDERS: Final[list[str]] = [
"cerebras",
"dashscope",
"snowflake",
+ "deepinfra",
]
@@ -221,6 +222,7 @@ PROVIDER_ALIASES: Final[dict[str, str]] = {
"cerebras": "cerebras",
"dashscope": "dashscope",
"snowflake": "snowflake",
+ "deepinfra": "deepinfra",
}
@@ -444,6 +446,12 @@ class LLM(BaseLLM):
# OpenRouter uses org/model format but accepts anything
return True
+ if provider == "deepinfra":
+ # DeepInfra ids are exactly org/model, so a full reference is
+ # deepinfra/<org>/<model>: one slash with both parts non-empty.
+ parts = model_lower.split("/")
+ return len(parts) == 2 and all(parts)
+
if provider == "snowflake":
return True
@@ -458,7 +466,8 @@ class LLM(BaseLLM):
a ``<prefix>/<model>`` string goes native when the prefix is a native
provider and the model is one it knows (or, for ``openai/``, when a
custom endpoint is configured); a bare model name infers its provider.
- ``native_class`` is ``None`` when the call falls back to LiteLLM.
+ ``native_class`` is ``None`` when the call falls back to LiteLLM; a
+ DeepInfra id that is not ``org/model`` raises instead of falling through.
``kwargs`` is read, never mutated.
"""
custom_openai = bool(kwargs.get("custom_openai", False))
@@ -481,10 +490,16 @@ class LLM(BaseLLM):
provider = explicit_provider
use_native = True
model_string = model
+ if provider == "deepinfra":
+ cls._require_deepinfra_model(model)
elif "/" in model:
prefix, _, model_part = model.partition("/")
canonical_provider = PROVIDER_ALIASES.get(prefix.lower())
+ if canonical_provider == "deepinfra":
+ # Raise here rather than fall through to LiteLLM, where the same
+ # malformed id fails later and differently per environment.
+ cls._require_deepinfra_model(model_part, prefix=f"{prefix}/")
valid_native_model = bool(
canonical_provider
@@ -514,6 +529,20 @@ class LLM(BaseLLM):
native_class = None
return _Route(provider, model_string, native_class, custom_openai_route)
+ @classmethod
+ def _require_deepinfra_model(cls, model: str, prefix: str = "") -> None:
+ """Raise unless ``model`` is the ``org/model`` form DeepInfra serves.
+
+ ``prefix`` is the ``deepinfra/`` the caller wrote before ``model``, if
+ any, so the example and the echoed value match what they passed.
+ """
+ if cls._matches_provider_pattern(model, "deepinfra"):
+ return
+ raise ValueError(
+ "DeepInfra model ids are org/model, for example "
+ f"'{prefix}deepseek-ai/DeepSeek-V4-Flash-0731'; got '{prefix}{model}'"
+ )
+
@classmethod
def _validate_model_in_constants(cls, model: str, provider: str) -> bool:
"""Validate if a model name exists in the provider's constants or matches provider patterns.
@@ -650,6 +679,7 @@ class LLM(BaseLLM):
"hosted_vllm",
"cerebras",
"dashscope",
+ "deepinfra",
}
if provider in openai_compatible_providers:
from crewai.llms.providers.openai_compatible.completion import (
diff --git a/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py b/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py
index fcb3f810c..4e2aa85f7 100644
--- a/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py
+++ b/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py
@@ -90,6 +90,12 @@ OPENAI_COMPATIBLE_PROVIDERS: dict[str, ProviderConfig] = {
base_url_env="DASHSCOPE_BASE_URL",
api_key_required=True,
),
+ "deepinfra": ProviderConfig(
+ base_url="https://api.deepinfra.com/v1/openai",
+ api_key_env="DEEPINFRA_API_KEY",
+ base_url_env="DEEPINFRA_BASE_URL",
+ api_key_required=True,
+ ),
}
_OLLAMA_DEFAULT_PORT = 11434
@@ -142,6 +148,7 @@ class OpenAICompatibleCompletion(OpenAICompletion):
- hosted_vllm: vLLM server (https://github.com/vllm-project/vllm)
- cerebras: Cerebras (https://cerebras.ai)
- dashscope: Alibaba Dashscope/Qwen (https://dashscope.aliyun.com)
+ - deepinfra: DeepInfra (https://deepinfra.com)
Example:
# Using provider prefix
diff --git a/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py b/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py
index 49106050b..fee26a7a9 100644
--- a/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py
+++ b/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py
@@ -95,6 +95,14 @@ class TestProviderRegistry:
assert config.api_key_env == "DASHSCOPE_API_KEY"
assert config.api_key_required is True
+ def test_deepinfra_config(self):
+ """Test DeepInfra provider configuration."""
+ config = OPENAI_COMPATIBLE_PROVIDERS["deepinfra"]
+ assert config.base_url == "https://api.deepinfra.com/v1/openai"
+ assert config.api_key_env == "DEEPINFRA_API_KEY"
+ assert config.base_url_env == "DEEPINFRA_BASE_URL"
+ assert config.api_key_required is True
+
class TestNormalizeOllamaBaseUrl:
"""Tests for _normalize_ollama_base_url helper."""
@@ -162,6 +170,15 @@ class TestOpenAICompatibleCompletion:
if original is not None:
os.environ[env_key] = original
+ def test_deepinfra_missing_api_key_names_env_var(self):
+ """DeepInfra requires a key; the error tells the user which env var to set."""
+ with patch.dict(os.environ, {}, clear=True), pytest.raises(
+ ValueError, match="DEEPINFRA_API_KEY"
+ ):
+ OpenAICompatibleCompletion(
+ model="deepseek-ai/DeepSeek-V4-Flash-0731", provider="deepinfra"
+ )
+
def test_api_key_from_env(self):
"""Test API key is read from environment variable."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key-from-env"}):
@@ -312,6 +329,87 @@ class TestLLMIntegration:
assert llm.provider == "dashscope"
assert llm.base_url == "https://my-dashscope.example.com/v1"
+ def test_llm_creates_openai_compatible_for_deepinfra(self):
+ """Test LLM factory creates OpenAICompatibleCompletion for DeepInfra."""
+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}):
+ llm = LLM(model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731")
+ assert isinstance(llm, OpenAICompatibleCompletion)
+ assert llm.provider == "deepinfra"
+ # DeepInfra ids are org/model, so the part after the provider
+ # prefix is itself two segments and must reach the API intact.
+ assert llm.model == "deepseek-ai/DeepSeek-V4-Flash-0731"
+ assert llm.base_url == "https://api.deepinfra.com/v1/openai"
+
+ def test_deepinfra_base_url_env_override(self):
+ """DEEPINFRA_BASE_URL redirects DeepInfra to a proxy or private endpoint."""
+ with patch.dict(
+ os.environ,
+ {
+ "DEEPINFRA_API_KEY": "test-key",
+ "DEEPINFRA_BASE_URL": "https://proxy.example.com/v1/openai",
+ },
+ ):
+ llm = LLM(model="deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct")
+ assert isinstance(llm, OpenAICompatibleCompletion)
+ assert llm.base_url == "https://proxy.example.com/v1/openai"
+
+ def test_llm_creates_openai_compatible_for_deepinfra_explicit_provider(self):
+ """Explicit provider="deepinfra" routes natively without the model prefix."""
+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}):
+ llm = LLM(model="deepseek-ai/DeepSeek-V4-Flash-0731", provider="deepinfra")
+ assert isinstance(llm, OpenAICompatibleCompletion)
+ assert llm.provider == "deepinfra"
+ assert llm.model == "deepseek-ai/DeepSeek-V4-Flash-0731"
+
+ @pytest.mark.parametrize(
+ "model",
+ [
+ "deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",
+ "DeepSeek-V4-Flash-0731",
+ "deepseek-ai/DeepSeek-V4-Flash-0731/",
+ ],
+ )
+ def test_llm_explicit_deepinfra_provider_rejects_malformed_model(self, model):
+ """provider="deepinfra" applies the org/model check before building a client."""
+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}), pytest.raises(
+ ValueError, match="org/model"
+ ):
+ LLM(model=model, provider="deepinfra")
+
+ @pytest.mark.parametrize(
+ "model",
+ [
+ "deepinfra/DeepSeek-V4-Flash-0731",
+ "deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731/",
+ "deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731/extra",
+ ],
+ )
+ def test_llm_rejects_malformed_deepinfra_prefixed_model(self, model):
+ """A malformed deepinfra/... id raises the org/model error, never reaching LiteLLM."""
+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}), pytest.raises(
+ ValueError, match="org/model"
+ ) as exc_info:
+ LLM(model=model)
+ assert model in str(exc_info.value)
+
+ @pytest.mark.parametrize(
+ ("model", "expected"),
+ [
+ ("deepseek-ai/DeepSeek-V4-Flash-0731", True),
+ ("Qwen/Qwen3.5-27B", True),
+ ("meta-llama/Llama-4-Scout-17B-16E-Instruct", True),
+ ("", False),
+ ("qwen", False),
+ ("Qwen/", False),
+ ("/Qwen3.5-27B", False),
+ ("deepseek-ai/DeepSeek-V4-Flash-0731/extra", False),
+ ("deepseek-ai/DeepSeek-V4-Flash-0731/", False),
+ ],
+ )
+ def test_deepinfra_requires_an_org_and_a_model(self, model, expected):
+ """DeepInfra references are exactly org/model: one slash, both parts present."""
+ assert LLM._matches_provider_pattern(model, "deepinfra") is expected
+
def test_llm_with_explicit_provider(self):
"""Test LLM with explicit provider parameter."""
with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key"}):