Skip to content

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

updating docs

João Moura854c58ded7ef
diff --git a/docs/core-concepts/LLMs.md b/docs/core-concepts/LLMs.md
new file mode 100644
index 000000000..7ae3e6948
--- /dev/null
+++ b/docs/core-concepts/LLMs.md
@@ -0,0 +1,155 @@
+# Large Language Models (LLMs) in crewAI
+
+## Introduction
+Large Language Models (LLMs) are the backbone of intelligent agents in the crewAI framework. This guide will help you understand, configure, and optimize LLM usage for your crewAI projects.
+
+## Table of Contents
+- [Key Concepts](#key-concepts)
+- [Configuring LLMs for Agents](#configuring-llms-for-agents)
+ - [1. Default Configuration](#1-default-configuration)
+ - [2. String Identifier](#2-string-identifier)
+ - [3. LLM Instance](#3-llm-instance)
+ - [4. Custom LLM Objects](#4-custom-llm-objects)
+- [Connecting to OpenAI-Compatible LLMs](#connecting-to-openai-compatible-llms)
+- [LLM Configuration Options](#llm-configuration-options)
+- [Using Ollama (Local LLMs)](#using-ollama-local-llms)
+- [Changing the Base API URL](#changing-the-base-api-url)
+- [Best Practices](#best-practices)
+- [Troubleshooting](#troubleshooting)
+
+## Key Concepts
+- **LLM**: Large Language Model, the AI powering agent intelligence
+- **Agent**: A crewAI entity that uses an LLM to perform tasks
+- **Provider**: A service that offers LLM capabilities (e.g., OpenAI, Anthropic, Ollama, [more providers](https://docs.litellm.ai/docs/providers))
+
+## Configuring LLMs for Agents
+
+crewAI offers flexible options for setting up LLMs:
+
+### 1. Default Configuration
+By default, crewAI uses the `gpt-4o-mini` model. It uses environment variables if no LLM is specified:
+- `OPENAI_MODEL_NAME` (defaults to "gpt-4o-mini" if not set)
+- `OPENAI_API_BASE`
+- `OPENAI_API_KEY`
+
+### 2. String Identifier
+```python
+agent = Agent(llm="gpt-4o", ...)
+```
+
+### 3. LLM Instance
+List of [more providers](https://docs.litellm.ai/docs/providers).
+```python
+from crewai import LLM
+
+llm = LLM(model="gpt-4", temperature=0.7)
+agent = Agent(llm=llm, ...)
+```
+
+### 4. Custom LLM Objects
+Pass a custom LLM implementation or object from another library.
+
+## Connecting to OpenAI-Compatible LLMs
+
+You can connect to OpenAI-compatible LLMs using either environment variables or by setting specific attributes on the LLM class:
+
+1. Using environment variables:
+```python
+import os
+
+os.environ["OPENAI_API_KEY"] = "your-api-key"
+os.environ["OPENAI_API_BASE"] = "https://api.your-provider.com/v1"
+```
+
+2. Using LLM class attributes:
+```python
+llm = LLM(
+ model="custom-model-name",
+ api_key="your-api-key",
+ base_url="https://api.your-provider.com/v1"
+)
+agent = Agent(llm=llm, ...)
+```
+
+## LLM Configuration Options
+
+When configuring an LLM for your agent, you have access to a wide range of parameters:
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `model` | str | The name of the model to use (e.g., "gpt-4", "gpt-3.5-turbo", "ollama/llama3.1", [more providers](https://docs.litellm.ai/docs/providers)) |
+| `timeout` | float, int | Maximum time (in seconds) to wait for a response |
+| `temperature` | float | Controls randomness in output (0.0 to 1.0) |
+| `top_p` | float | Controls diversity of output (0.0 to 1.0) |
+| `n` | int | Number of completions to generate |
+| `stop` | str, List[str] | Sequence(s) to stop generation |
+| `max_tokens` | int | Maximum number of tokens to generate |
+| `presence_penalty` | float | Penalizes new tokens based on their presence in the text so far |
+| `frequency_penalty` | float | Penalizes new tokens based on their frequency in the text so far |
+| `logit_bias` | Dict[int, float] | Modifies likelihood of specified tokens appearing in the completion |
+| `response_format` | Dict[str, Any] | Specifies the format of the response (e.g., {"type": "json_object"}) |
+| `seed` | int | Sets a random seed for deterministic results |
+| `logprobs` | bool | Whether to return log probabilities of the output tokens |
+| `top_logprobs` | int | Number of most likely tokens to return the log probabilities for |
+| `base_url` | str | The base URL for the API endpoint |
+| `api_version` | str | The version of the API to use |
+| `api_key` | str | Your API key for authentication |
+
+Example:
+```python
+llm = LLM(
+ model="gpt-4",
+ temperature=0.8,
+ max_tokens=150,
+ top_p=0.9,
+ frequency_penalty=0.1,
+ presence_penalty=0.1,
+ stop=["END"],
+ seed=42,
+ base_url="https://api.openai.com/v1",
+ api_key="your-api-key-here"
+)
+agent = Agent(llm=llm, ...)
+```
+
+## Using Ollama (Local LLMs)
+
+crewAI supports using Ollama for running open-source models locally:
+
+1. Install Ollama: [ollama.ai](https://ollama.ai/)
+2. Run a model: `ollama run llama2`
+3. Configure agent:
+```python
+agent = Agent(
+ llm=LLM(model="ollama/llama3.1", base_url="http://localhost:11434"),
+ ...
+)
+```
+
+## Changing the Base API URL
+
+You can change the base API URL for any LLM provider by setting the `base_url` parameter:
+
+```python
+llm = LLM(
+ model="custom-model-name",
+ base_url="https://api.your-provider.com/v1",
+ api_key="your-api-key"
+)
+agent = Agent(llm=llm, ...)
+```
+
+This is particularly useful when working with OpenAI-compatible APIs or when you need to specify a different endpoint for your chosen provider.
+
+## Best Practices
+1. **Choose the right model**: Balance capability and cost.
+2. **Optimize prompts**: Clear, concise instructions improve output.
+3. **Manage tokens**: Monitor and limit token usage for efficiency.
+4. **Use appropriate temperature**: Lower for factual tasks, higher for creative ones.
+5. **Implement error handling**: Gracefully manage API errors and rate limits.
+
+## Troubleshooting
+- **API Errors**: Check your API key, network connection, and rate limits.
+- **Unexpected Outputs**: Refine your prompts and adjust temperature or top_p.
+- **Performance Issues**: Consider using a more powerful model or optimizing your queries.
+- **Timeout Errors**: Increase the `timeout` parameter or optimize your input.
\ No newline at end of file
diff --git a/docs/how-to/LLM-Connections.md b/docs/how-to/LLM-Connections.md
index 11d214ac0..d1cbd8c8c 100644
--- a/docs/how-to/LLM-Connections.md
+++ b/docs/how-to/LLM-Connections.md
@@ -5,10 +5,10 @@ description: Comprehensive guide on integrating CrewAI with various Large Langua
## Connect CrewAI to LLMs
-CrewAI now uses LiteLLM to connect to a wide variety of Language Models (LLMs). This integration provides extensive versatility, allowing you to use models from numerous providers with a simple, unified interface.
+CrewAI uses LiteLLM to connect to a wide variety of Language Models (LLMs). This integration provides extensive versatility, allowing you to use models from numerous providers with a simple, unified interface.
!!! note "Default LLM"
- By default, CrewAI uses OpenAI's GPT-4 model (specifically, the model specified by the OPENAI_MODEL_NAME environment variable, defaulting to "gpt-4") for language processing. You can easily configure your agents to use a different model or provider as described in this guide.
+ By default, CrewAI uses the `gpt-4o-mini` model. This is determined by the `OPENAI_MODEL_NAME` environment variable, which defaults to "gpt-4o-mini" if not set. You can easily configure your agents to use a different model or provider as described in this guide.
## Supported Providers
@@ -35,7 +35,11 @@ For a complete and up-to-date list of supported providers, please refer to the [
## Changing the LLM
-To use a different LLM with your CrewAI agents, you simply need to pass the model name as a string when initializing the agent. Here are some examples:
+To use a different LLM with your CrewAI agents, you have several options:
+
+### 1. Using a String Identifier
+
+Pass the model name as a string when initializing the agent:
```python
from crewai import Agent
@@ -55,59 +59,105 @@ claude_agent = Agent(
backstory="An AI assistant leveraging Anthropic's language model.",
llm='claude-2'
)
+```
-# Using Ollama's local Llama 2 model
-ollama_agent = Agent(
- role='Local AI Expert',
- goal='Process information using a local model',
- backstory="An AI assistant running on local hardware.",
- llm='ollama/llama2'
+### 2. Using the LLM Class
+
+For more detailed configuration, use the LLM class:
+
+```python
+from crewai import Agent, LLM
+
+llm = LLM(
+ model="gpt-4",
+ temperature=0.7,
+ base_url="https://api.openai.com/v1",
+ api_key="your-api-key-here"
)
-# Using Google's Gemini model
-gemini_agent = Agent(
- role='Google AI Expert',
- goal='Generate creative content with Gemini',
- backstory="An AI assistant powered by Google's advanced language model.",
- llm='gemini-pro'
+agent = Agent(
+ role='Customized LLM Expert',
+ goal='Provide tailored responses',
+ backstory="An AI assistant with custom LLM settings.",
+ llm=llm
)
```
-## Configuration
+## Configuration Options
-For most providers, you'll need to set up your API keys as environment variables. Here's how you can do it for some common providers:
+When configuring an LLM for your agent, you have access to a wide range of parameters:
-```python
-import os
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `model` | str | The name of the model to use (e.g., "gpt-4", "claude-2") |
+| `temperature` | float | Controls randomness in output (0.0 to 1.0) |
+| `max_tokens` | int | Maximum number of tokens to generate |
+| `top_p` | float | Controls diversity of output (0.0 to 1.0) |
+| `frequency_penalty` | float | Penalizes new tokens based on their frequency in the text so far |
+| `presence_penalty` | float | Penalizes new tokens based on their presence in the text so far |
+| `stop` | str, List[str] | Sequence(s) to stop generation |
+| `base_url` | str | The base URL for the API endpoint |
+| `api_key` | str | Your API key for authentication |
+
+For a complete list of parameters and their descriptions, refer to the LLM class documentation.
-# OpenAI
-os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
+## Connecting to OpenAI-Compatible LLMs
-# Anthropic
-os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-api-key"
+You can connect to OpenAI-compatible LLMs using either environment variables or by setting specific attributes on the LLM class:
-# Google (Vertex AI)
-os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "path/to/your/credentials.json"
+### Using Environment Variables
-# Azure OpenAI
-os.environ["AZURE_API_KEY"] = "your-azure-api-key"
-os.environ["AZURE_API_BASE"] = "your-azure-endpoint"
+```python
+import os
-# AWS (Bedrock)
-os.environ["AWS_ACCESS_KEY_ID"] = "your-aws-access-key-id"
-os.environ["AWS_SECRET_ACCESS_KEY"] = "your-aws-secret-access-key"
+os.environ["OPENAI_API_KEY"] = "your-api-key"
+os.environ["OPENAI_API_BASE"] = "https://api.your-provider.com/v1"
+os.environ["OPENAI_MODEL_NAME"] = "your-model-name"
```
-For providers that require additional configuration or have specific setup requirements, please refer to the [LiteLLM documentation](https://docs.litellm.ai/docs/) for detailed instructions.
+### Using LLM Class Attributes
-## Using Local Models
+```python
+llm = LLM(
+ model="custom-model-name",
+ api_key="your-api-key",
+ base_url="https://api.your-provider.com/v1"
+)
+agent = Agent(llm=llm, ...)
+```
+
+## Using Local Models with Ollama
-For local models like those provided by Ollama, ensure you have the necessary software installed and running. For example, to use Ollama:
+For local models like those provided by Ollama:
1. [Download and install Ollama](https://ollama.com/download)
2. Pull the desired model (e.g., `ollama pull llama2`)
-3. Use the model in your CrewAI agent by specifying `llm='ollama/llama2'`
+3. Configure your agent:
+
+```python
+agent = Agent(
+ role='Local AI Expert',
+ goal='Process information using a local model',
+ backstory="An AI assistant running on local hardware.",
+ llm=LLM(model="ollama/llama2", base_url="http://localhost:11434")
+)
+```
+
+## Changing the Base API URL
+
+You can change the base API URL for any LLM provider by setting the `base_url` parameter:
+
+```python
+llm = LLM(
+ model="custom-model-name",
+ base_url="https://api.your-provider.com/v1",
+ api_key="your-api-key"
+)
+agent = Agent(llm=llm, ...)
+```
+
+This is particularly useful when working with OpenAI-compatible APIs or when you need to specify a different endpoint for your chosen provider.
## Conclusion
-By leveraging LiteLLM, CrewAI now offers seamless integration with a vast array of LLMs. This flexibility allows you to choose the most suitable model for your specific needs, whether you prioritize performance, cost-efficiency, or local deployment. Remember to consult the [LiteLLM documentation](https://docs.litellm.ai/docs/) for the most up-to-date information on supported models and configuration options.
\ No newline at end of file
+By leveraging LiteLLM, CrewAI offers seamless integration with a vast array of LLMs. This flexibility allows you to choose the most suitable model for your specific needs, whether you prioritize performance, cost-efficiency, or local deployment. Remember to consult the [LiteLLM documentation](https://docs.litellm.ai/docs/) for the most up-to-date information on supported models and configuration options.
\ No newline at end of file
diff --git a/docs/index.md b/docs/index.md
index 3f8917dac..1674d6e22 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -53,6 +53,11 @@ Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By
Crews
</a>
</li>
+ <li>
+ <a href="./core-concepts/LLMs">
+ LLMs
+ </a>
+ </li>
<li>
<a href="./core-concepts/Pipeline">
Pipeline