Skip to content

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

[Feat]: Add OpenTelemetry Instrumentation for monitoring FireCrawl (#547)

Patcherffdfeed6b96d
diff --git a/docs/images/docs-firecrawl-banner.gif b/docs/images/docs-firecrawl-banner.gif
new file mode 100644
index 00000000..4af88e9e
Binary files /dev/null and b/docs/images/docs-firecrawl-banner.gif differ
diff --git a/docs/latest/integrations/firecrawl.mdx b/docs/latest/integrations/firecrawl.mdx
new file mode 100644
index 00000000..be4becb0
--- /dev/null
+++ b/docs/latest/integrations/firecrawl.mdx
@@ -0,0 +1,78 @@
+---
+title: 'Monitor FireCrawl using OpenTelemetry'
+sidebarTitle: 'FireCrawl'
+---
+
+<Frame>
+ <img src="/images/docs-firecrawl-banner.gif" />
+</Frame>
+
+OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor crawling Agents built using FireCrawl. This includes tracking performance and operation stats.
+
+Auto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.
+
+The integration is compatible with
+- FireCrawl Python SDK client `>= 1.6.3`
+
+## Get Started
+
+<Steps>
+ <Step title="Install OpenLIT">
+ Open your command line or terminal and run:
+ ```shell
+ pip install openlit
+ ```
+ </Step>
+ <Step title="Initialize OpenLIT in your Application">
+ You can set up OpenLIT in your application using either function arguments directly in your code or by using environment variables.
+ <Tabs>
+ <Tab title="Setup using function arguments">
+
+ Add the following two lines to your application code:
+
+ ```python
+ import openlit
+
+ openlit.init(
+ otlp_endpoint="YOUR_OTEL_ENDPOINT",
+ )
+ ```
+
+ Replace:
+ 1. `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.
+ </Tab>
+ <Tab title="Setup using Environment Variables">
+
+ Add the following two lines to your application code:
+
+ ```python
+ import openlit
+
+ openlit.init()
+ ```
+
+ Then, configure the your OTLP endpoint using environment variable:
+ ```shell
+ export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_OTEL_ENDPOINT"
+ ```
+
+ Replace:
+ 1. `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.
+ </Tab>
+ </Tabs>
+ To send metrics and traces to other Observability tools, refer to the [Connections Guide](/latest/connections/intro).
+
+ For more advanced configurations and application use cases, visit the [OpenLIT Python repository](https://github.com/openlit/openlit/tree/main/sdk/python).
+ </Step>
+</Steps>
+
+---
+
+<CardGroup cols={2}>
+<Card title="Connections" href="/latest/connections/intro" icon='link'>
+Connect to your existing Observablity Stack
+</Card>
+<Card title="SDK configuration" href="/latest/sdk-configuration" icon='file-code'>
+Documentation of the configuration options for the OpenLIT SDK.
+</Card>
+</CardGroup>
\ No newline at end of file
diff --git a/docs/latest/integrations/introduction.mdx b/docs/latest/integrations/introduction.mdx
index acfa8daf..2dd7deec 100644
--- a/docs/latest/integrations/introduction.mdx
+++ b/docs/latest/integrations/introduction.mdx
@@ -2193,6 +2193,13 @@ Start integrating and monitoring your applications with OpenLIT in just one line
icon="robot"
>
</Card>
+ <Card
+ title="FireCrawl"
+ href="/latest/integrations/firecrawl"
+ iconType="duotone"
+ icon="fire"
+ >
+ </Card>
<Card
title="Dynamiq"
href="/latest/integrations/dynamiq"
diff --git a/docs/mint.json b/docs/mint.json
index 66bace23..3b7fa13a 100644
--- a/docs/mint.json
+++ b/docs/mint.json
@@ -139,6 +139,7 @@
"latest/integrations/multion",
"latest/integrations/julep-ai",
"latest/integrations/crawl4ai",
+ "latest/integrations/firecrawl",
"latest/integrations/dynamiq",
"latest/integrations/controlflow"
]
diff --git a/sdk/python/README.md b/sdk/python/README.md
index 3c9820c1..86cfe4a1 100644
--- a/sdk/python/README.md
+++ b/sdk/python/README.md
@@ -50,7 +50,7 @@ This project proudly follows and maintains the [Semantic Conventions](https://gi
| [✅ Groq](https://docs.openlit.io/latest/integrations/groq) | | [✅ Julep AI](https://docs.openlit.io/latest/integrations/julep-ai) | |
| [✅ ElevenLabs](https://docs.openlit.io/latest/integrations/elevenlabs) | | [✅ ControlFlow](https://docs.openlit.io/latest/integrations/controlflow) | |
| [✅ vLLM](https://docs.openlit.io/latest/integrations/vllm) | | [✅ Crawl4AI](https://docs.openlit.io/latest/integrations/crawl4ai) | |
-| [✅ OLA Krutrim](https://docs.openlit.io/latest/integrations/krutrim) | | | |
+| [✅ OLA Krutrim](https://docs.openlit.io/latest/integrations/krutrim) | | [✅ FireCrawl](https://docs.openlit.io/latest/integrations/firecrawl) | |
| [✅ Google AI Studio](https://docs.openlit.io/latest/integrations/google-ai-studio) | | | |
| [✅ NVIDIA NIM](https://docs.openlit.io/latest/integrations/nvidia-nim) | | | |
| [✅ Titan ML](https://docs.openlit.io/latest/integrations/titan-ml) | | | |
diff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.toml
index fed23a9e..51abb71b 100644
--- a/sdk/python/pyproject.toml
+++ b/sdk/python/pyproject.toml
@@ -1,6 +1,6 @@
[tool.poetry]
name = "openlit"
-version = "1.32.10"
+version = "1.32.11"
description = "OpenTelemetry-native Auto instrumentation library for monitoring LLM Applications and GPUs, facilitating the integration of observability into your GenAI-driven projects"
authors = ["OpenLIT"]
repository = "https://github.com/openlit/openlit/tree/main/openlit/python"
diff --git a/sdk/python/src/openlit/__init__.py b/sdk/python/src/openlit/__init__.py
index e2248689..f61da198 100644
--- a/sdk/python/src/openlit/__init__.py
+++ b/sdk/python/src/openlit/__init__.py
@@ -61,6 +61,7 @@ from openlit.instrumentation.julep import JulepInstrumentor
from openlit.instrumentation.ai21 import AI21Instrumentor
from openlit.instrumentation.controlflow import ControlFlowInstrumentor
from openlit.instrumentation.crawl4ai import Crawl4AIInstrumentor
+from openlit.instrumentation.firecrawl import FireCrawlInstrumentor
from openlit.instrumentation.gpu import GPUInstrumentor
import openlit.guard
import openlit.evals
@@ -260,6 +261,7 @@ def init(environment="default", application_name="default", tracer=None, otlp_en
"controlflow": "controlflow",
"assemblyai": "assemblyai",
"crawl4ai": "crawl4ai",
+ "firecrawl": "firecrawl",
}
invalid_instrumentors = [
@@ -354,6 +356,7 @@ def init(environment="default", application_name="default", tracer=None, otlp_en
"controlflow": ControlFlowInstrumentor(),
"assemblyai": AssemblyAIInstrumentor(),
"crawl4ai": Crawl4AIInstrumentor(),
+ "firecrawl": FireCrawlInstrumentor(),
}
# Initialize and instrument only the enabled instrumentors
diff --git a/sdk/python/src/openlit/instrumentation/crawl4ai/async_crawl4ai.py b/sdk/python/src/openlit/instrumentation/crawl4ai/async_crawl4ai.py
index 5054c313..eab21a59 100644
--- a/sdk/python/src/openlit/instrumentation/crawl4ai/async_crawl4ai.py
+++ b/sdk/python/src/openlit/instrumentation/crawl4ai/async_crawl4ai.py
@@ -67,7 +67,7 @@ def async_wrap_crawl(gen_ai_endpoint, version, environment, application_name,
span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
environment)
span.set_attribute(SemanticConvetion.GEN_AI_AGENT_TYPE,
- "browser")
+ SemanticConvetion.GEN_AI_AGENT_TYPE_BROWSER)
span.set_attribute(SemanticConvetion.GEN_AI_AGENT_ENABLE_CACHE, not kwargs.get("disable_cache", False))
url = kwargs.get("url") if "url" in kwargs else str(args[0]) if args else None
diff --git a/sdk/python/src/openlit/instrumentation/crawl4ai/crawl4ai.py b/sdk/python/src/openlit/instrumentation/crawl4ai/crawl4ai.py
index 3f6f0b53..82b67e4c 100644
--- a/sdk/python/src/openlit/instrumentation/crawl4ai/crawl4ai.py
+++ b/sdk/python/src/openlit/instrumentation/crawl4ai/crawl4ai.py
@@ -67,7 +67,7 @@ def wrap_crawl(gen_ai_endpoint, version, environment, application_name,
span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
environment)
span.set_attribute(SemanticConvetion.GEN_AI_AGENT_TYPE,
- "browser")
+ SemanticConvetion.GEN_AI_AGENT_TYPE_BROWSER)
span.set_attribute(SemanticConvetion.GEN_AI_AGENT_ENABLE_CACHE, not kwargs.get("disable_cache", False))
url = kwargs.get("url") if "url" in kwargs else str(args[0]) if args else None
diff --git a/sdk/python/src/openlit/instrumentation/firecrawl/__init__.py b/sdk/python/src/openlit/instrumentation/firecrawl/__init__.py
new file mode 100644
index 00000000..bf6bf4c8
--- /dev/null
+++ b/sdk/python/src/openlit/instrumentation/firecrawl/__init__.py
@@ -0,0 +1,49 @@
+# pylint: disable=useless-return, bad-staticmethod-argument, disable=duplicate-code
+"""Initializer of Auto Instrumentation of FireCrawl Functions"""
+
+from typing import Collection
+import importlib.metadata
+from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
+from wrapt import wrap_function_wrapper
+
+from openlit.instrumentation.firecrawl.firecrawl import (
+ wrap_crawl
+)
+
+_instruments = ("firecrawl-py >= 1.6.3",)
+
+class FireCrawlInstrumentor(BaseInstrumentor):
+ """
+ An instrumentor for firecrawl's client library.
+ """
+
+ def instrumentation_dependencies(self) -> Collection[str]:
+ return _instruments
+
+ def _instrument(self, **kwargs):
+ application_name = kwargs.get("application_name", "default_application")
+ environment = kwargs.get("environment", "default_environment")
+ tracer = kwargs.get("tracer")
+ metrics = kwargs.get("metrics_dict")
+ pricing_info = kwargs.get("pricing_info", {})
+ trace_content = kwargs.get("trace_content", False)
+ disable_metrics = kwargs.get("disable_metrics")
+ version = importlib.metadata.version("firecrawl-py")
+
+ wrap_function_wrapper(
+ "firecrawl.firecrawl",
+ "FirecrawlApp.scrape_url",
+ wrap_crawl("firecrawl.scrape_url", version, environment, application_name,
+ tracer, pricing_info, trace_content, metrics, disable_metrics),
+ )
+
+ wrap_function_wrapper(
+ "firecrawl.firecrawl",
+ "FirecrawlApp.crawl_url",
+ wrap_crawl("firecrawl.crawl_url", version, environment, application_name,
+ tracer, pricing_info, trace_content, metrics, disable_metrics),
+ )
+
+ def _uninstrument(self, **kwargs):
+ # Proper uninstrumentation logic to revert patched methods
+ pass
diff --git a/sdk/python/src/openlit/instrumentation/firecrawl/firecrawl.py b/sdk/python/src/openlit/instrumentation/firecrawl/firecrawl.py
new file mode 100644
index 00000000..9e0f2f66
--- /dev/null
+++ b/sdk/python/src/openlit/instrumentation/firecrawl/firecrawl.py
@@ -0,0 +1,90 @@
+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument
+"""
+Module for monitoring FireCrawl calls.
+"""
+
+import logging
+from opentelemetry.trace import SpanKind, Status, StatusCode
+from opentelemetry.sdk.resources import TELEMETRY_SDK_NAME
+from openlit.__helpers import (
+ handle_exception,
+)
+from openlit.semcov import SemanticConvetion
+
+# Initialize logger for logging potential issues and operations
+logger = logging.getLogger(__name__)
+
+def wrap_crawl(gen_ai_endpoint, version, environment, application_name,
+ tracer, pricing_info, trace_content, metrics, disable_metrics):
+ """
+ Generates a telemetry wrapper for chat completions to collect metrics.
+
+ Args:
+ gen_ai_endpoint: Endpoint identifier for logging and tracing.
+ version: Version of the monitoring package.
+ environment: Deployment environment (e.g., production, staging).
+ application_name: Name of the application using the FireCrawl Agent.
+ tracer: OpenTelemetry tracer for creating spans.
+ pricing_info: Information used for calculating the cost of FireCrawl usage.
+ trace_content: Flag indicating whether to trace the actual content.
+
+ Returns:
+ A function that wraps the chat completions method to add telemetry.
+ """
+
+ def wrapper(wrapped, instance, args, kwargs):
+ """
+ Wraps the API call to add telemetry.
+
+ This collects metrics such as execution time, cost, and token usage, and handles errors
+ gracefully, adding details to the trace for observability.
+
+ Args:
+ wrapped: The original method to be wrapped.
+ instance: The instance of the class where the original method is defined.
+ args: Positional arguments for the method.
+ kwargs: Keyword arguments for the method.
+
+ Returns:
+ The response from the original method.
+ """
+
+ # pylint: disable=line-too-long
+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:
+ response = wrapped(*args, **kwargs)
+
+ try:
+ # Set base span attribues
+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")
+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,
+ SemanticConvetion.GEN_AI_SYSTEM_FIRECRAWL)
+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,
+ SemanticConvetion.GEN_AI_TYPE_AGENT)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,
+ gen_ai_endpoint)
+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,
+ application_name)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
+ environment)
+ span.set_attribute(SemanticConvetion.GEN_AI_AGENT_TYPE,
+ SemanticConvetion.GEN_AI_AGENT_TYPE_BROWSER)
+ span.set_attribute(SemanticConvetion.GEN_AI_AGENT_PARAMS,
+ str(kwargs.get("params")))
+
+ url = kwargs.get("url") if "url" in kwargs else str(args[0]) if args else None
+ if url is not None:
+ span.set_attribute(SemanticConvetion.GEN_AI_AGENT_BROWSE_URL, url)
+
+ span.set_status(Status(StatusCode.OK))
+
+ # Return original response
+ return response
+
+ except Exception as e:
+ handle_exception(span, e)
+ logger.error("Error in trace creation: %s", e)
+
+ # Return original response
+ return response
+
+ return wrapper
diff --git a/sdk/python/src/openlit/semcov/__init__.py b/sdk/python/src/openlit/semcov/__init__.py
index e70945c9..047afe78 100644
--- a/sdk/python/src/openlit/semcov/__init__.py
+++ b/sdk/python/src/openlit/semcov/__init__.py
@@ -132,6 +132,7 @@ class SemanticConvetion:
GEN_AI_SYSTEM_CONTROLFLOW = "controlflow"
GEN_AI_SYSTEM_ASSEMBLYAI = "assemblyai"
GEN_AI_SYSTEM_CRAWL4AI = "crawl4ai"
+ GEN_AI_SYSTEM_FIRECRAWL = "firecrawl"
# Vector DB
DB_OPERATION_API_ENDPOINT = "db.operation.api_endpoint"
@@ -200,6 +201,7 @@ class SemanticConvetion:
GEN_AI_AGENT_TOOLS = "gen_ai.agent.tools"
GEN_AI_AGENT_TOOL_RESULTS = "gen_ai.agent.tool_results"
GEN_AI_AGENT_TASK = "gen_ai.agent.task"
+ GEN_AI_AGENT_PARAMS = "gen_ai.agent.params"
GEN_AI_AGENT_INSTRUCTIONS = "gen_ai.agent.instructions"
GEN_AI_AGENT_STORAGE = "gen_ai.agent.storage"
GEN_AI_AGENT_EXPECTED_OUTPUT = "gen_ai.agent.expected_output"
@@ -212,6 +214,8 @@ class SemanticConvetion:
GEN_AI_AGENT_RESPONSE_TIME = "gen_ai.agent.response_time"
GEN_AI_AGENT_STRATEGY = "gen_ai.agent.strategy"
+ GEN_AI_AGENT_TYPE_BROWSER = "browser"
+
# GPU
GPU_INDEX = "gpu.index"
GPU_UUID = "gpu.uuid"
diff --git a/sdk/python/tests/requirements.txt b/sdk/python/tests/requirements.txt
index 5f0ad469..79e0ace8 100644
--- a/sdk/python/tests/requirements.txt
+++ b/sdk/python/tests/requirements.txt
@@ -25,6 +25,7 @@ julep
ai21
controlflow
assemblyai
+firecrawl-py
# Vector DBs
chromadb
diff --git a/sdk/python/tests/test_firecrawl.py b/sdk/python/tests/test_firecrawl.py
new file mode 100644
index 00000000..3b8813ed
--- /dev/null
+++ b/sdk/python/tests/test_firecrawl.py
@@ -0,0 +1,38 @@
+# pylint: disable=duplicate-code, no-name-in-module, import-error
+"""
+This module contains tests for FireCrawl functionality using the FireCrawl Python library.
+
+Tests cover various API endpoints, including chat and embeddings.
+These tests validate integration with OpenLIT.
+
+Environment Variables:
+ - FIRECRAWL_API_KEY: FireCrawl API key for authentication.
+
+Note: Ensure the environment is properly configured for FireCrawl access and OpenLIT monitoring
+prior to running these tests.
+"""
+
+import os
+from firecrawl import FirecrawlApp
+import openlit
+
+# Initialize synchronous FireCrawl client
+sync_client = FirecrawlApp(api_key=os.getenv("FIRECRAWL_API_KEY"))
+
+# Initialize environment and application name for OpenLIT monitoring
+openlit.init(environment="openlit-python-testing", application_name="openlit-python-firecrawl-test")
+
+def test_sync_scarpe_url():
+ """
+ Tests synchronous scrape url.
+
+ Raises:
+ AssertionError: If the response object is not as expected.
+ """
+
+ response = sync_client.scrape_url(
+ 'https://openlit.io',
+ params={'formats': ['markdown', 'html'],}
+ )
+
+ assert isinstance(response, dict)