Skip to content

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

Create opik-observability.mdx

Abby Morgan5b7fed2cb647
diff --git a/docs/how-to/opik-observability.mdx b/docs/how-to/opik-observability.mdx
new file mode 100644
index 000000000..49e0c6495
--- /dev/null
+++ b/docs/how-to/opik-observability.mdx
@@ -0,0 +1,116 @@
+---
+title: Opik Integration
+description: Learn how to use Comet Opik to debug, evaluate, and monitor your CrewAI applications with comprehensive tracing, automated evaluations, and production-ready dashboards.
+icon: insights
+---
+
+# Opik Overview
+
+With [Comet Opik](https://www.comet.com/docs/opik/), debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
+
+Opik provides comprehensive support for every stage of your CrewAI application development:
+
+- **Log Traces and Spans**: Automatically track LLM calls and application logic to debug and analyze development and production systems. Manually or programmatically annotate, view, and compare responses across projects.
+- **Evaluate Your LLM Application's Performance**: Evaluate against a custom test set and run built-in evaluation metrics or define your own metrics in the SDK or UI.
+- **Test Within Your CI/CD Pipeline**: Establish reliable performance baselines with Opik's LLM unit tests, built on PyTest. Run online evaluations for continuous monitoring in production.
+- **Monitor & Analyze Production Data**: Understand your models' performance on unseen data in production and generate datasets for new dev iterations.
+
+## Setup
+Comet provides a hosted version of the Opik platform, or you can run the platform locally.
+
+To use the hosted version, simply [create a Comet account](https://www.comet.com/signup?utm_medium=github&utm_source=crewai_docs) and grab you API Key.
+
+To run the Opik platform locally, see our [installation guide](https://www.comet.com/docs/opik/self-host/overview/) for more information.
+
+For this guide we will use CrewAI’s quickstart example.
+
+<Steps>
+ <Step title="Install required packages">
+ ```shell
+ %pip install crewai crewai-tools opik --upgrade
+ ```
+ </Step>
+ <Step title="Configure Opik">
+ ```python
+ import opik
+ opik.configure(use_local=False)
+ ```
+ <Step title="Prepare environment">
+ First, we set up our API keys for our LLM-provider as environment variables:
+ ```python
+ import os
+ import getpass
+
+ if "OPENAI_API_KEY" not in os.environ:
+ os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")
+ ```
+ </Step>
+ <Step title="Using CrewAI">
+ The first step is to create our project. We will use an example from CrewAI’s documentation:
+ ```python
+ from crewai import Agent, Crew, Task, Process
+
+
+class YourCrewName:
+ def agent_one(self) -> Agent:
+ return Agent(
+ role="Data Analyst",
+ goal="Analyze data trends in the market",
+ backstory="An experienced data analyst with a background in economics",
+ verbose=True,
+ )
+
+ def agent_two(self) -> Agent:
+ return Agent(
+ role="Market Researcher",
+ goal="Gather information on market dynamics",
+ backstory="A diligent researcher with a keen eye for detail",
+ verbose=True,
+ )
+
+ def task_one(self) -> Task:
+ return Task(
+ name="Collect Data Task",
+ description="Collect recent market data and identify trends.",
+ expected_output="A report summarizing key trends in the market.",
+ agent=self.agent_one(),
+ )
+
+ def task_two(self) -> Task:
+ return Task(
+ name="Market Research Task",
+ description="Research factors affecting market dynamics.",
+ expected_output="An analysis of factors influencing the market.",
+ agent=self.agent_two(),
+ )
+
+ def crew(self) -> Crew:
+ return Crew(
+ agents=[self.agent_one(), self.agent_two()],
+ tasks=[self.task_one(), self.task_two()],
+ process=Process.sequential,
+ verbose=True,
+ )
+
+ ```
+ Now we can import Opik’s tracker and run our crew:
+ ```python
+ from opik.integrations.crewai import track_crewai
+
+ track_crewai(project_name="crewai-integration-demo")
+
+ my_crew = YourCrewName().crew()
+ result = my_crew.kickoff()
+
+ print(result)
+ ```
+ After running your CrewAI application, visit the Opik app to view:
+ - LLM traces, spans, and their metadata
+ - Agent interactions and task execution flow
+ - Performance metrics like latency and token usage
+ - Evaluation metrics (built-in or custom)
+
+ <Frame caption="Opik Agent Dashboard">
+ <img src="/images/opik-crewai-dashboard.png" alt="Opik agent monitoring example with CrewAI" />
+ </Frame>
+<\Steps>