Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
feat sambanova models (#1858)
Jorge Piedrahita Ortiz0e942367353a
diff --git a/docs/concepts/cli.mdx b/docs/concepts/cli.mdxindex ab21232ad..ad64ee938 100644--- a/docs/concepts/cli.mdx+++ b/docs/concepts/cli.mdx@@ -161,6 +161,7 @@ The CLI will initially prompt for API keys for the following services:* Groq* Anthropic* Google Gemini+* SambaNovaWhen you select a provider, the CLI will prompt you to enter your API key.diff --git a/docs/concepts/llms.mdx b/docs/concepts/llms.mdxindex 4e269d71a..851e93085 100644--- a/docs/concepts/llms.mdx+++ b/docs/concepts/llms.mdx@@ -146,6 +146,19 @@ Here's a detailed breakdown of supported models and their capabilities, you canGroq is known for its fast inference speeds, making it suitable for real-time applications.</Tip></Tab>+ <Tab title="SambaNova">+ | Model | Context Window | Best For |+ |-------|---------------|-----------|+ | Llama 3.1 70B/8B | Up to 131,072 tokens | High-performance, large context tasks |+ | Llama 3.1 405B | 8,192 tokens | High-performance and output quality |+ | Llama 3.2 Series | 8,192 tokens | General-purpose tasks, multimodal |+ | Llama 3.3 70B | Up to 131,072 tokens | High-performance and output quality|+ | Qwen2 familly | 8,192 tokens | High-performance and output quality |++ <Tip>+ [SambaNova](https://cloud.sambanova.ai/) has several models with fast inference speed at full precision.+ </Tip>+ </Tab><Tab title="Others">| Provider | Context Window | Key Features ||----------|---------------|--------------|diff --git a/docs/how-to/llm-connections.mdx b/docs/how-to/llm-connections.mdxindex a69aa4c86..25509c299 100644--- a/docs/how-to/llm-connections.mdx+++ b/docs/how-to/llm-connections.mdx@@ -32,6 +32,7 @@ LiteLLM supports a wide range of providers, including but not limited to:- Cloudflare Workers AI- DeepInfra- Groq+- SambaNova- [NVIDIA NIMs](https://docs.api.nvidia.com/nim/reference/models-1)- And many more!diff --git a/src/crewai/cli/constants.py b/src/crewai/cli/constants.pyindex 2d3c5bdad..d878c6f34 100644--- a/src/crewai/cli/constants.py+++ b/src/crewai/cli/constants.py@@ -85,6 +85,12 @@ ENV_VARS = {"key_name": "CEREBRAS_API_KEY",},],+ "sambanova": [+ {+ "prompt": "Enter your SambaNovaCloud API key (press Enter to skip)",+ "key_name": "SAMBANOVA_API_KEY",+ }+ ],}@@ -98,6 +104,7 @@ PROVIDERS = ["bedrock","azure","cerebras",+ "sambanova",]MODELS = {@@ -156,6 +163,19 @@ MODELS = {"bedrock/mistral.mistral-7b-instruct-v0:2","bedrock/mistral.mixtral-8x7b-instruct-v0:1",],+ "sambanova": [+ "sambanova/Meta-Llama-3.3-70B-Instruct",+ "sambanova/QwQ-32B-Preview",+ "sambanova/Qwen2.5-72B-Instruct",+ "sambanova/Qwen2.5-Coder-32B-Instruct",+ "sambanova/Meta-Llama-3.1-405B-Instruct",+ "sambanova/Meta-Llama-3.1-70B-Instruct",+ "sambanova/Meta-Llama-3.1-8B-Instruct",+ "sambanova/Llama-3.2-90B-Vision-Instruct",+ "sambanova/Llama-3.2-11B-Vision-Instruct",+ "sambanova/Meta-Llama-3.2-3B-Instruct",+ "sambanova/Meta-Llama-3.2-1B-Instruct",+ ],}DEFAULT_LLM_MODEL = "gpt-4o-mini"diff --git a/src/crewai/llm.py b/src/crewai/llm.pyindex 085a0abdc..74a6bc2c3 100644--- a/src/crewai/llm.py+++ b/src/crewai/llm.py@@ -76,6 +76,18 @@ LLM_CONTEXT_WINDOW_SIZES = {"mixtral-8x7b-32768": 32768,"llama-3.3-70b-versatile": 128000,"llama-3.3-70b-instruct": 128000,+ #sambanova+ "Meta-Llama-3.3-70B-Instruct": 131072,+ "QwQ-32B-Preview": 8192,+ "Qwen2.5-72B-Instruct": 8192,+ "Qwen2.5-Coder-32B-Instruct": 8192,+ "Meta-Llama-3.1-405B-Instruct": 8192,+ "Meta-Llama-3.1-70B-Instruct": 131072,+ "Meta-Llama-3.1-8B-Instruct": 131072,+ "Llama-3.2-90B-Vision-Instruct": 16384,+ "Llama-3.2-11B-Vision-Instruct": 16384,+ "Meta-Llama-3.2-3B-Instruct": 4096,+ "Meta-Llama-3.2-1B-Instruct": 16384,}DEFAULT_CONTEXT_WINDOW_SIZE = 8192