Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
update readme and docs (#199)
patcher9dfa6d710a5bb
diff --git a/README.md b/README.mdindex c0dc7b44..7b1385e5 100644--- a/README.md+++ b/README.md@@ -31,7 +31,14 @@ LIT stands for Learning Interpretability Tool. It refers to a visual, interactiv## 🚀 Getting Started-## Step 1: Install OpenLIT SDK+## Step 1: Deploy OpenLIT++From the root directory of the this Repo, Run the below command:+```shell+docker-compose up -d+```++## Step 2: Install OpenLIT SDK```bashpip install openlit@@ -43,10 +50,12 @@ Integrating the OpenLIT into LLM applications is straightforward. Start monitori```pythonimport openlit-openlit.init()+openlit.init(otlp_endpoint="http://127.0.0.1:4318")```-By default, OpenLIT directs traces and metrics straight to your console. To forward telemetry data to an HTTP OTLP endpoint, such as the OpenTelemetry Collector, set the `otlp_endpoint` parameter with the desired endpoint. Alternatively, you can configure the endpoint by setting the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable as recommended in the OpenTelemetry documentation.+To forward telemetry data to an HTTP OTLP endpoint, such as the OpenTelemetry Collector, set the `otlp_endpoint` parameter with the desired endpoint. Alternatively, you can configure the endpoint by setting the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable as recommended in the OpenTelemetry documentation.++> 💡 Info: If you dont provide `otlp_endpoint` function argument or set the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable, The SDK directs the trace directly to your console, which can be useful during development.To send telemetry to OpenTelemetry backends requiring authentication, set the `otlp_headers` parameter with its desired value. Alternatively, you can configure the endpoint by setting the `OTEL_EXPORTER_OTLP_HEADERS` environment variable as recommended in the OpenTelemetry documentation.@@ -73,19 +82,12 @@ export OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Basic%20<base64 encoded Insta```### Step 3: Visualize and Optimize!-With the LLM Observability data now being collected and sent to your chosen OpenTelemetry backend, the next step is to visualize and analyze this data to glean insights into your application's performance, behavior, and identify areas of improvement. Here is how you would use the data in Grafana, follow these detailed instructions to explore your LLM application's Telemetry data.-- - Select the **Explore** option from Grafana's sidebar.- - At the top, ensure the correct Tempo data source is selected from the dropdown menu.- - Use the **Query** field to specify any particular traces you are interested in, or leave it empty to browse through all the available traces.- - You can adjust the time range to focus on specific periods of interest.- - Hit **Run Query** to fetch your trace data. You'll see a visual representation of your traces along with detailed information on particular spans when clicked.+With the LLM Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your LLM application's performance, behavior, and identify areas of improvement.-#### Next Steps+Just head over to OpenLIT UI at `127.0.0.1:3000` on your browser to start exploring.-- **Create Dashboards:** Beyond just exploring traces, consider creating dashboards in Grafana to monitor key performance indicators (KPIs) and metrics over time. Dashboards can be customized with various panels to display graphs, logs, and single stats that are most relevant to your application's performance and usage patterns.-- **Set Alerts:** Grafana also allows you to set up alerts based on specific thresholds. This feature can be invaluable in proactively managing your application's health by notifying you of potential issues before they impact users.-- **Iterate and Optimize:** Use the insights gained from your observability data to make informed decisions on optimizing your LLM application. This might involve refining model parameters, adjusting scaling strategies, or identifying and resolving bottlenecks.++## 🌱 Contributingdiff --git a/docs/latest/configuration.mdx b/docs/latest/configuration.mdxindex 302ce47e..db7ec9d5 100644--- a/docs/latest/configuration.mdx+++ b/docs/latest/configuration.mdx@@ -4,12 +4,12 @@ description: 'Configuring Options for OpenLIT'icon: 'sliders'----This guide covers all the available environment variables to fine-tune OpenLIT according to your needs.+This guide covers all the available environment variables to fine-tune OpenLIT UI according to your needs.## Environment Varibales<ResponseField name="INIT_DB_HOST" required>- Sets the host address of the ClickHouse server for OpenLIT Client to connect+ Sets the host address of the ClickHouse server for OpenLIT UI to connect**Example**:```yaml@@ -27,7 +27,7 @@ This guide covers all the available environment variables to fine-tune OpenLIT a</ResponseField><ResponseField name="INIT_DB_DATABASE" required>- Sets the name of the database in Clickhouse to be used by OpenLIT+ Sets the name of the database in Clickhouse to be used by OpenLIT UI**Example**:```yamldiff --git a/docs/latest/introduction.mdx b/docs/latest/introduction.mdxindex d97aa115..d94e27bd 100644--- a/docs/latest/introduction.mdx+++ b/docs/latest/introduction.mdx@@ -1,6 +1,6 @@---title: What is OpenLIT?-description: 'OpenTelemetry-native GenAI and LLM Application Observability'+description: 'OpenTelemetry-native LLM Application Observability'---<Frame>@@ -15,6 +15,10 @@ This project proudly follows the [Semantic Conventions](https://github.com/open-LIT stands for Learning Interpretability Tool. It refers to a visual, interactive model-understanding and data visualization tool ad a term introduced by [Google](https://developers.google.com/machine-learning/glossary#learning-interpretability-tool-lit).## ⚡ Features+<Frame>+ <img src="https://github.com/openlit/.github/blob/main/profile/assets/github-readme-repo-banner.png?raw=true" />+</Frame>+- **OpenTelemetry-native**: Native support ensures that integrating OpenLIT into your projects feels more like a natural extension rather than an additional layer of complexity.- **Granular Usage Insights of your LLM Applications**: Assess your LLM's performance and costs with fine-grained control, breaking down metrics by environment (such as staging or production) or application, to optimize for efficiency and scalability.- **Vendor-Neutral SDKs**: In the spirit of OpenTelemetry, OpenLIT's SDKs are agnostic of the backend vendors. This means you can confidently use OpenLIT with various telemetry backends, like Grafana Tempo, without worrying about compatibility issues.diff --git a/docs/latest/quickstart.mdx b/docs/latest/quickstart.mdxindex 2db14e76..081c817a 100644--- a/docs/latest/quickstart.mdx+++ b/docs/latest/quickstart.mdx@@ -4,10 +4,25 @@ description: 'Quickly start **monitoring** your **LLM Applications** in just **aicon: 'bolt'----This guide will walk you through setting up OpenTelemetry Auto Instrumentation for monitoring your LLM Application. In just a few steps, you'll be able to track and analyze the performance and usage of your GenAI and LLM Applications.+This guide will walk you through setting up OpenTelemetry Auto Instrumentation for monitoring your LLM Application using OpenLIT. In just a few steps, you'll be able to track and analyze the performance and usage of your LLM Applications.In this guide, we'll show how you can send OpenTelemetry traces and metrics from your LLM Applications to Grafana Cloud<Steps>+ <Step title="Deploy OpenLIT">+ <Steps>+ <Step title="Git Clone OpenLIT Repository">+ ```shell+ git clone [email protected]:openlit/openlit.git+ ```+ </Step>+ <Step title="Start Docker Compose">+ From the root directory of the [OpenLIT Repo](https://github.com/openlit/openlit), Run the below command:+ ```shell+ docker-compose up -d+ ```+ </Step>+ </Steps>+ </Step><Step title="Install the OpenLIT Python SDK using pip">```shellpip install openlit@@ -15,23 +30,47 @@ In this guide, we'll show how you can send OpenTelemetry traces and metrics from</Step><Step title="Add the following two lines to your application code:">Add the snippet below to your Python application.-+ <Tabs>+ <Tab title="Setup using function arguments">```python- openlit.init(- otlp_endpoint="https://otlp-gateway-prod-us-east-0.grafana.net/otlp",- otlp_headers="Authorization=Basic%20<base64 encoded Instance ID and API Token>"- )+ import openlit++ openlit.init(otlp_endpoint="http://127.0.0.1:4318")```- Alternatively, You can also choose to set these values using `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` environment variables+ Example Usage for monitoring `OpenAI` Usage:++ ```python+ from openai import OpenAI+ import openlit++ openlit.init(otlp_endpoint="http://127.0.0.1:4318")++ client = OpenAI(+ api_key="YOUR_OPENAI_KEY"+ )+ chat_completion = client.chat.completions.create(+ messages=[+ {+ "role": "user",+ "content": "What is LLM Observability",+ }+ ],+ model="gpt-3.5-turbo",+ )+ ```+ </Tab>+ <Tab title="Setup using Environment Variables">```python+ import openlit+openlit.init()```- ```env- export OTEL_EXPORTER_OTLP_ENDPOINT = "https://otlp-gateway-prod-us-east-0.grafana.net/otlp"- export OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Basic%20<base64 encoded Instance ID and API Token>"+ Run the following command to configure the OTEL export endpoint:+ ```shell+ export OTEL_EXPORTER_OTLP_ENDPOINT = "http://127.0.0.1:4318"```Example Usage for monitoring `OpenAI` Usage:@@ -56,21 +95,22 @@ In this guide, we'll show how you can send OpenTelemetry traces and metrics frommodel="gpt-3.5-turbo",)```-+ </Tab>+ </Tabs>Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.</Step>- <Step title="Visualize and Analyze in Grafana">- With the LLM Observability data now being collected and sent to your chosen OpenTelemetry backend, the next step is to visualize and analyze this data to glean insights into your application's performance, behavior, and identify areas of improvement. Here is how you would use the data in Grafana, follow these detailed instructions to explore your LLM application's Telemetry data.-- - Select the **Explore** option from Grafana's sidebar.- - At the top, ensure the correct Tempo data source is selected from the dropdown menu.- - Use the **Query** field to specify any particular traces you are interested in, or leave it empty to browse through all the available traces.- - You can adjust the time range to focus on specific periods of interest.- - Hit **Run Query** to fetch your trace data. You'll see a visual representation of your traces along with detailed information on particular spans when clicked.+ <Step title="Visualize and Analyze">+ With the LLM Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your LLM application's performance, behavior, and identify areas of improvement.++ Just head over to OpenLIT UI at `127.0.0.1:3000` on your browser to start exploring.+ <Frame>+ <img src="https://github.com/openlit/.github/blob/main/profile/assets/openlit-client-1.png?raw=true" />+ <img src="https://github.com/openlit/.github/blob/main/profile/assets/openlit-client-2.png?raw=true" />+ </Frame></Step></Steps>-You're all set! Following these steps should have you on your way to effectively monitoring your LLM applications.+You're all set! Following these steps should have you on your way to effectively monitoring your LLM applications with OpenTelemetry. If you wish to send telemetry to any other backend, Refer to our [Connections](./connections/intro).If you have any questions or need support, reach out to our [community](https://join.slack.com/t/openlit/shared_invite/zt-2etnfttwg-TjP_7BZXfYg84oAukY8QRQ).diff --git a/docs/mint.json b/docs/mint.jsonindex d4330e02..054a06f0 100644--- a/docs/mint.json+++ b/docs/mint.json@@ -45,7 +45,8 @@"group": "Introduction","pages": ["latest/introduction",- "latest/quickstart"+ "latest/quickstart",+ "latest/configuration"],"version": "latest"},