Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
feat(llms): add deepinfra as an openai-compatible provider (#7459)
Georgi Atsevd38ceafb68c6
diff --git a/docs/edge/ar/concepts/llms.mdx b/docs/edge/ar/concepts/llms.mdxindex 863a80ee3..6da174f9e 100644--- a/docs/edge/ar/concepts/llms.mdx+++ b/docs/edge/ar/concepts/llms.mdx@@ -1012,6 +1012,39 @@ mode: "wide"uv add 'crewai[litellm]'```</Accordion>++ <Accordion title="DeepInfra">+ يتصل CrewAI بـ DeepInfra عبر عميل OpenAI الأصلي، لذا لا حاجة لتثبيت LiteLLM.++ عيّن متغيرات البيئة التالية في ملف `.env`:+ ```toml Code+ # Required+ DEEPINFRA_API_KEY=<your-api-key>++ # Optional, point at a proxy or a private deployment endpoint+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai+ ```++ مثال الاستخدام في مشروع CrewAI:+ ```python Code+ llm = LLM(+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",+ temperature=0.7+ )+ ```++ <Info>+ ميزات DeepInfra:+ - نماذج مفتوحة الأوزان (Llama وQwen وDeepSeek وGLM وMistral وغيرها) خلف API واحد متوافق مع OpenAI+ - استدعاء الأدوات والبث والمخرجات المنظمة بمخطط JSON+ - تسعير لكل token وطول السياق منشوران في كتالوج النماذج+ </Info>++ معرّفات النماذج بصيغة `org/model` أصلًا، لذا يتكوّن المرجع الكامل من ثلاثة أجزاء،+ مثل `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. راجع+ [كتالوج النماذج المنسّق لـ CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)،+ وهو عام ولا يتطلب مصادقة، للحصول على المعرّفات الحالية وأطوال السياق والأسعار.+ </Accordion></AccordionGroup>## بث الاستجاباتdiff --git a/docs/edge/en/concepts/llms.mdx b/docs/edge/en/concepts/llms.mdxindex 957a52e94..084c60082 100644--- a/docs/edge/en/concepts/llms.mdx+++ b/docs/edge/en/concepts/llms.mdx@@ -1155,6 +1155,39 @@ In this section, you'll find detailed examples that help you select, configure,uv add 'crewai[litellm]'```</Accordion>++ <Accordion title="DeepInfra">+ CrewAI talks to DeepInfra through its native OpenAI client, so no LiteLLM install is needed.++ Set the following environment variables in your `.env` file:+ ```toml Code+ # Required+ DEEPINFRA_API_KEY=<your-api-key>++ # Optional, point at a proxy or a private deployment endpoint+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai+ ```++ Example usage in your CrewAI project:+ ```python Code+ llm = LLM(+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",+ temperature=0.7+ )+ ```++ <Info>+ DeepInfra features:+ - Open-weight models (Llama, Qwen, DeepSeek, GLM, Mistral and more) behind one OpenAI-compatible API+ - Tool calling, streaming and JSON schema structured outputs+ - Per-token pricing and context length published in the model catalog+ </Info>++ Model ids are already in `org/model` form, so a full reference carries three segments,+ for example `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. See the+ [model catalog curated for CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),+ which is public and needs no authentication, for current ids, context lengths and pricing.+ </Accordion></AccordionGroup>## Streaming Responsesdiff --git a/docs/edge/ko/concepts/llms.mdx b/docs/edge/ko/concepts/llms.mdxindex 63b3f861c..df477da70 100644--- a/docs/edge/ko/concepts/llms.mdx+++ b/docs/edge/ko/concepts/llms.mdx@@ -755,6 +755,39 @@ CrewAI는 고유한 기능, 인증 방법, 모델 역량을 제공하는 다양uv add 'crewai[litellm]'```</Accordion>++ <Accordion title="DeepInfra">+ CrewAI는 네이티브 OpenAI 클라이언트로 DeepInfra에 연결하므로 LiteLLM 설치가 필요하지 않습니다.++ `.env` 파일에 다음 환경 변수를 설정하십시오:+ ```toml Code+ # Required+ DEEPINFRA_API_KEY=<your-api-key>++ # Optional, point at a proxy or a private deployment endpoint+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai+ ```++ CrewAI 프로젝트에서의 예시 사용법:+ ```python Code+ llm = LLM(+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",+ temperature=0.7+ )+ ```++ <Info>+ DeepInfra 특징:+ - 하나의 OpenAI 호환 API로 제공되는 오픈 웨이트 모델 (Llama, Qwen, DeepSeek, GLM, Mistral 등)+ - 도구 호출, 스트리밍, JSON 스키마 구조화 출력 지원+ - 모델 카탈로그에 공개된 토큰 단위 가격과 컨텍스트 길이+ </Info>++ 모델 ID는 이미 `org/model` 형식이므로 전체 참조는 세 개의 세그먼트로 구성됩니다.+ 예: `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. 현재 ID, 컨텍스트 길이, 가격은+ 인증 없이 공개된 [CrewAI용으로 선별된 모델 카탈로그](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai)를+ 참조하세요.+ </Accordion></AccordionGroup>## 스트리밍 응답diff --git a/docs/edge/pt-BR/concepts/llms.mdx b/docs/edge/pt-BR/concepts/llms.mdxindex 45ef20bbc..ce46c01dd 100644--- a/docs/edge/pt-BR/concepts/llms.mdx+++ b/docs/edge/pt-BR/concepts/llms.mdx@@ -728,6 +728,39 @@ Nesta seção, você encontrará exemplos detalhados que ajudam a selecionar, couv add 'crewai[litellm]'```</Accordion>++ <Accordion title="DeepInfra">+ O CrewAI se comunica com o DeepInfra pelo cliente OpenAI nativo, então não é necessário instalar o LiteLLM.++ Defina as seguintes variáveis de ambiente no seu arquivo `.env`:+ ```toml Code+ # Required+ DEEPINFRA_API_KEY=<your-api-key>++ # Optional, point at a proxy or a private deployment endpoint+ DEEPINFRA_BASE_URL=https://api.deepinfra.com/v1/openai+ ```++ Exemplo de uso em seu projeto CrewAI:+ ```python Code+ llm = LLM(+ model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",+ temperature=0.7+ )+ ```++ <Info>+ Recursos do DeepInfra:+ - Modelos de pesos abertos (Llama, Qwen, DeepSeek, GLM, Mistral e outros) por trás de uma única API compatível com OpenAI+ - Chamada de ferramentas, streaming e saídas estruturadas com JSON schema+ - Preço por token e tamanho de contexto publicados no catálogo de modelos+ </Info>++ Os IDs dos modelos já estão no formato `org/model`, então uma referência completa tem três segmentos,+ por exemplo `deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct`. Consulte o+ [catálogo de modelos selecionado para o CrewAI](https://api.deepinfra.com/v1/openai/models?filter=with_meta&sort_by=crewai),+ que é público e não exige autenticação, para obter os IDs atuais, tamanhos de contexto e preços.+ </Accordion></AccordionGroup>## Respostas em streamingdiff --git a/lib/crewai/src/crewai/llm.py b/lib/crewai/src/crewai/llm.pyindex 2ba1e6409..fff8f7e27 100644--- a/lib/crewai/src/crewai/llm.py+++ b/lib/crewai/src/crewai/llm.py@@ -197,6 +197,7 @@ SUPPORTED_NATIVE_PROVIDERS: Final[list[str]] = ["cerebras","dashscope","snowflake",+ "deepinfra",]@@ -221,6 +222,7 @@ PROVIDER_ALIASES: Final[dict[str, str]] = {"cerebras": "cerebras","dashscope": "dashscope","snowflake": "snowflake",+ "deepinfra": "deepinfra",}@@ -444,6 +446,12 @@ class LLM(BaseLLM):# OpenRouter uses org/model format but accepts anythingreturn True+ if provider == "deepinfra":+ # DeepInfra ids are exactly org/model, so a full reference is+ # deepinfra/<org>/<model>: one slash with both parts non-empty.+ parts = model_lower.split("/")+ return len(parts) == 2 and all(parts)+if provider == "snowflake":return True@@ -458,7 +466,8 @@ class LLM(BaseLLM):a ``<prefix>/<model>`` string goes native when the prefix is a nativeprovider and the model is one it knows (or, for ``openai/``, when acustom endpoint is configured); a bare model name infers its provider.- ``native_class`` is ``None`` when the call falls back to LiteLLM.+ ``native_class`` is ``None`` when the call falls back to LiteLLM; a+ DeepInfra id that is not ``org/model`` raises instead of falling through.``kwargs`` is read, never mutated."""custom_openai = bool(kwargs.get("custom_openai", False))@@ -481,10 +490,16 @@ class LLM(BaseLLM):provider = explicit_provideruse_native = Truemodel_string = model+ if provider == "deepinfra":+ cls._require_deepinfra_model(model)elif "/" in model:prefix, _, model_part = model.partition("/")canonical_provider = PROVIDER_ALIASES.get(prefix.lower())+ if canonical_provider == "deepinfra":+ # Raise here rather than fall through to LiteLLM, where the same+ # malformed id fails later and differently per environment.+ cls._require_deepinfra_model(model_part, prefix=f"{prefix}/")valid_native_model = bool(canonical_provider@@ -514,6 +529,20 @@ class LLM(BaseLLM):native_class = Nonereturn _Route(provider, model_string, native_class, custom_openai_route)+ @classmethod+ def _require_deepinfra_model(cls, model: str, prefix: str = "") -> None:+ """Raise unless ``model`` is the ``org/model`` form DeepInfra serves.++ ``prefix`` is the ``deepinfra/`` the caller wrote before ``model``, if+ any, so the example and the echoed value match what they passed.+ """+ if cls._matches_provider_pattern(model, "deepinfra"):+ return+ raise ValueError(+ "DeepInfra model ids are org/model, for example "+ f"'{prefix}deepseek-ai/DeepSeek-V4-Flash-0731'; got '{prefix}{model}'"+ )+@classmethoddef _validate_model_in_constants(cls, model: str, provider: str) -> bool:"""Validate if a model name exists in the provider's constants or matches provider patterns.@@ -650,6 +679,7 @@ class LLM(BaseLLM):"hosted_vllm","cerebras","dashscope",+ "deepinfra",}if provider in openai_compatible_providers:from crewai.llms.providers.openai_compatible.completion import (diff --git a/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py b/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.pyindex fcb3f810c..4e2aa85f7 100644--- a/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py+++ b/lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py@@ -90,6 +90,12 @@ OPENAI_COMPATIBLE_PROVIDERS: dict[str, ProviderConfig] = {base_url_env="DASHSCOPE_BASE_URL",api_key_required=True,),+ "deepinfra": ProviderConfig(+ base_url="https://api.deepinfra.com/v1/openai",+ api_key_env="DEEPINFRA_API_KEY",+ base_url_env="DEEPINFRA_BASE_URL",+ api_key_required=True,+ ),}_OLLAMA_DEFAULT_PORT = 11434@@ -142,6 +148,7 @@ class OpenAICompatibleCompletion(OpenAICompletion):- hosted_vllm: vLLM server (https://github.com/vllm-project/vllm)- cerebras: Cerebras (https://cerebras.ai)- dashscope: Alibaba Dashscope/Qwen (https://dashscope.aliyun.com)+ - deepinfra: DeepInfra (https://deepinfra.com)Example:# Using provider prefixdiff --git a/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py b/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.pyindex 49106050b..fee26a7a9 100644--- a/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py+++ b/lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py@@ -95,6 +95,14 @@ class TestProviderRegistry:assert config.api_key_env == "DASHSCOPE_API_KEY"assert config.api_key_required is True+ def test_deepinfra_config(self):+ """Test DeepInfra provider configuration."""+ config = OPENAI_COMPATIBLE_PROVIDERS["deepinfra"]+ assert config.base_url == "https://api.deepinfra.com/v1/openai"+ assert config.api_key_env == "DEEPINFRA_API_KEY"+ assert config.base_url_env == "DEEPINFRA_BASE_URL"+ assert config.api_key_required is True+class TestNormalizeOllamaBaseUrl:"""Tests for _normalize_ollama_base_url helper."""@@ -162,6 +170,15 @@ class TestOpenAICompatibleCompletion:if original is not None:os.environ[env_key] = original+ def test_deepinfra_missing_api_key_names_env_var(self):+ """DeepInfra requires a key; the error tells the user which env var to set."""+ with patch.dict(os.environ, {}, clear=True), pytest.raises(+ ValueError, match="DEEPINFRA_API_KEY"+ ):+ OpenAICompatibleCompletion(+ model="deepseek-ai/DeepSeek-V4-Flash-0731", provider="deepinfra"+ )+def test_api_key_from_env(self):"""Test API key is read from environment variable."""with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key-from-env"}):@@ -312,6 +329,87 @@ class TestLLMIntegration:assert llm.provider == "dashscope"assert llm.base_url == "https://my-dashscope.example.com/v1"+ def test_llm_creates_openai_compatible_for_deepinfra(self):+ """Test LLM factory creates OpenAICompatibleCompletion for DeepInfra."""+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}):+ llm = LLM(model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731")+ assert isinstance(llm, OpenAICompatibleCompletion)+ assert llm.provider == "deepinfra"+ # DeepInfra ids are org/model, so the part after the provider+ # prefix is itself two segments and must reach the API intact.+ assert llm.model == "deepseek-ai/DeepSeek-V4-Flash-0731"+ assert llm.base_url == "https://api.deepinfra.com/v1/openai"++ def test_deepinfra_base_url_env_override(self):+ """DEEPINFRA_BASE_URL redirects DeepInfra to a proxy or private endpoint."""+ with patch.dict(+ os.environ,+ {+ "DEEPINFRA_API_KEY": "test-key",+ "DEEPINFRA_BASE_URL": "https://proxy.example.com/v1/openai",+ },+ ):+ llm = LLM(model="deepinfra/meta-llama/Llama-4-Scout-17B-16E-Instruct")+ assert isinstance(llm, OpenAICompatibleCompletion)+ assert llm.base_url == "https://proxy.example.com/v1/openai"++ def test_llm_creates_openai_compatible_for_deepinfra_explicit_provider(self):+ """Explicit provider="deepinfra" routes natively without the model prefix."""+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}):+ llm = LLM(model="deepseek-ai/DeepSeek-V4-Flash-0731", provider="deepinfra")+ assert isinstance(llm, OpenAICompatibleCompletion)+ assert llm.provider == "deepinfra"+ assert llm.model == "deepseek-ai/DeepSeek-V4-Flash-0731"++ @pytest.mark.parametrize(+ "model",+ [+ "deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731",+ "DeepSeek-V4-Flash-0731",+ "deepseek-ai/DeepSeek-V4-Flash-0731/",+ ],+ )+ def test_llm_explicit_deepinfra_provider_rejects_malformed_model(self, model):+ """provider="deepinfra" applies the org/model check before building a client."""+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}), pytest.raises(+ ValueError, match="org/model"+ ):+ LLM(model=model, provider="deepinfra")++ @pytest.mark.parametrize(+ "model",+ [+ "deepinfra/DeepSeek-V4-Flash-0731",+ "deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731/",+ "deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731/extra",+ ],+ )+ def test_llm_rejects_malformed_deepinfra_prefixed_model(self, model):+ """A malformed deepinfra/... id raises the org/model error, never reaching LiteLLM."""+ with patch.dict(os.environ, {"DEEPINFRA_API_KEY": "test-key"}), pytest.raises(+ ValueError, match="org/model"+ ) as exc_info:+ LLM(model=model)+ assert model in str(exc_info.value)++ @pytest.mark.parametrize(+ ("model", "expected"),+ [+ ("deepseek-ai/DeepSeek-V4-Flash-0731", True),+ ("Qwen/Qwen3.5-27B", True),+ ("meta-llama/Llama-4-Scout-17B-16E-Instruct", True),+ ("", False),+ ("qwen", False),+ ("Qwen/", False),+ ("/Qwen3.5-27B", False),+ ("deepseek-ai/DeepSeek-V4-Flash-0731/extra", False),+ ("deepseek-ai/DeepSeek-V4-Flash-0731/", False),+ ],+ )+ def test_deepinfra_requires_an_org_and_a_model(self, model, expected):+ """DeepInfra references are exactly org/model: one slash, both parts present."""+ assert LLM._matches_provider_pattern(model, "deepinfra") is expected+def test_llm_with_explicit_provider(self):"""Test LLM with explicit provider parameter."""with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "test-key"}):