Skip to content

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

[Feat]: Updating Documentation to include AMD GPU Monitoring and cosmetic changes (#497)

Patcher44089c7b0b37
diff --git a/amd-logo-pre-2013.svg b/amd-logo-pre-2013.svg
new file mode 100644
index 00000000..f0957aa9
--- /dev/null
+++ b/amd-logo-pre-2013.svg
@@ -0,0 +1 @@
+<svg height="596" width="2500" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 190.804"><path d="M187.888 178.12H143.52l-13.573-32.735H56.003L43.637 178.12H0L66.667 12.776h47.761zM91.155 52.285L66.912 116.53h50.913zM349.056 12.776h35.88V178.12h-41.219V74.842l-44.608 51.878h-6.301l-44.605-51.878V178.12h-41.219V12.776h35.88l53.093 61.336zM489.375 12.776c60.364 0 91.391 37.573 91.391 82.909 0 47.517-30.058 82.435-96 82.435h-68.369V12.776zm-31.762 135.039h26.906c41.457 0 53.823-28.127 53.823-52.375 0-28.368-15.276-52.363-54.308-52.363h-26.422v104.738z"/><g fill="#00a76d"><path d="M748.028 51.981h-85.259L610.797 0H800v189.21l-51.972-51.975zM662.708 137.296V62.397L609.2 115.901v74.903h74.889l53.505-53.508z"/></g></svg>
\ No newline at end of file
diff --git a/docs/images/docs-amd-banner.gif b/docs/images/docs-amd-banner.gif
new file mode 100644
index 00000000..9f07fe94
Binary files /dev/null and b/docs/images/docs-amd-banner.gif differ
diff --git a/docs/latest/connections/datadog.mdx b/docs/latest/connections/datadog.mdx
index 8c7a8b71..884c3796 100644
--- a/docs/latest/connections/datadog.mdx
+++ b/docs/latest/connections/datadog.mdx
@@ -3,7 +3,7 @@ title: 'DataDog'
description: 'LLM Observability with DataDog and OpenLIT'
---
-To send OpenTelemetry data collected by OpenLIt from your LLM Application, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
+To directly send OpenTelemetry data collected by OpenLIT SDK from your AI Application to DataDog, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
```python
import openlit
diff --git a/docs/latest/connections/dynatrace.mdx b/docs/latest/connections/dynatrace.mdx
index ef80d65c..9ed2aaa1 100644
--- a/docs/latest/connections/dynatrace.mdx
+++ b/docs/latest/connections/dynatrace.mdx
@@ -3,7 +3,7 @@ title: 'Dynatrace'
description: 'LLM Observability with Dynatrace and OpenLIT'
---
-To send OpenTelemetry data collected by OpenLIt from your LLM Application, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
+To directly send OpenTelemetry data collected by OpenLIT SDK from your AI Application to Dynatrace, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
```python
import openlit
diff --git a/docs/latest/connections/elastic.mdx b/docs/latest/connections/elastic.mdx
index 142b5f39..fd14a740 100644
--- a/docs/latest/connections/elastic.mdx
+++ b/docs/latest/connections/elastic.mdx
@@ -12,7 +12,7 @@ description: 'LLM Observability with Elastic and OpenLIT'
<img src="/images/elastic-dashboard-4.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to Elastic, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to Elastic, Follow the below steps.
<Steps>
<Step title="Get your Elastic APM Credentials">
<Steps>
diff --git a/docs/latest/connections/grafanacloud.mdx b/docs/latest/connections/grafanacloud.mdx
index 83bc01d8..cd2fa18e 100644
--- a/docs/latest/connections/grafanacloud.mdx
+++ b/docs/latest/connections/grafanacloud.mdx
@@ -8,7 +8,7 @@ description: 'LLM Observability with Grafana Cloud and OpenLIT'
<img src="/images/grafana-cloud-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to Grafana Cloud, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to Grafana Cloud, Follow the below steps.
<Steps>
<Step title="Get your Grafana Cloud Credentials">
<Steps>
diff --git a/docs/latest/connections/highlight.mdx b/docs/latest/connections/highlight.mdx
index 8a6ba942..82190655 100644
--- a/docs/latest/connections/highlight.mdx
+++ b/docs/latest/connections/highlight.mdx
@@ -3,7 +3,7 @@ title: 'Highlight.io'
description: 'LLM Observability with Highlight.io and OpenLIT'
---
-To send OpenTelemetry data collected by OpenLIt from your LLM Application, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
+To directly send OpenTelemetry data collected by OpenLIT SDK from your AI Application to Highlight.io, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
```python
import openlit
diff --git a/docs/latest/connections/hyperdx.mdx b/docs/latest/connections/hyperdx.mdx
index 3bfbc352..b82cd64b 100644
--- a/docs/latest/connections/hyperdx.mdx
+++ b/docs/latest/connections/hyperdx.mdx
@@ -11,7 +11,7 @@ description: 'LLM Observability with HyperDX using OpenLIT'
<Frame>
<img src="/images/hyperdx-dashboard-3.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to HyperDX, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to HyperDX, Follow the below steps.
<Steps>
<Step title="Install OpenTelemetry Collector (Optional)">
diff --git a/docs/latest/connections/intro.mdx b/docs/latest/connections/intro.mdx
index 16f6ccff..2c85358b 100644
--- a/docs/latest/connections/intro.mdx
+++ b/docs/latest/connections/intro.mdx
@@ -4,9 +4,19 @@ description: "Send LLM Monitoring data to your existing Observablity Stack"
sidebarTitle: "Introduction"
---
-OpenLIT enhances observability by seamlessly exporting processed LLM data to your observability stack. This assists developers in bringing together all application-related data, including LLM usage metrics, on a unified platform, simplifying the visualization and assessment of the entire application ecosystem.
+```mermaid
+flowchart TB;
+ subgraph " "
+ direction LR;
+ ApplicationCode[Application Code] -->|Instrumented with| OpenLIT_SDK[OpenLIT SDK];
+ OpenLIT_SDK -->|Sends Traces & Metrics| OT_Backend[OpenTelemetry Backend];
+ end
+```
-Below is a list of observability tools that work seamlessly with OpenLIT:
+
+The OpenLIT SDK can send processed execution traces and metrics from AI applications directly to your existing observability stack. This capability assists developers in consolidating all application-related data, including AI usage metrics, onto a unified platform, thereby simplifying the visualization and assessment of the entire AI application ecosystem.
+
+Below is a list of observability tools that work seamlessly with OpenLIT SDK:
<CardGroup cols={2}>
<Card
diff --git a/docs/latest/connections/middleware.mdx b/docs/latest/connections/middleware.mdx
index 32c64b17..9b40dadd 100644
--- a/docs/latest/connections/middleware.mdx
+++ b/docs/latest/connections/middleware.mdx
@@ -12,7 +12,7 @@ description: 'LLM Observability with Middleware and OpenLIT'
<img src="/images/middleware-dashboard-4.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to Middleware, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your LLM Application to Middleware, Follow the below steps.
<Steps>
<Step title="Get your Middleware credentials">
<Steps>
diff --git a/docs/latest/connections/murnitur.mdx b/docs/latest/connections/murnitur.mdx
index 450d430c..b4ee387e 100644
--- a/docs/latest/connections/murnitur.mdx
+++ b/docs/latest/connections/murnitur.mdx
@@ -3,7 +3,7 @@ title: "Murnitur"
description: "LLM Observability with Murnitur and OpenLIT"
---
-To send OpenTelemetry data collected by OpenLIt from your LLM Application, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
+To directly send OpenTelemetry data collected by OpenLIT from your AI Application to murnitur.ai, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
```python
import openlit
diff --git a/docs/latest/connections/new-relic.mdx b/docs/latest/connections/new-relic.mdx
index 6e8d4cdc..3cd18f89 100644
--- a/docs/latest/connections/new-relic.mdx
+++ b/docs/latest/connections/new-relic.mdx
@@ -7,7 +7,7 @@ description: 'LLM Observability with New Relic and OpenLIT'
<img src="/images/new-relic-dashboard-1.png" />
<img src="/images/new-relic-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to New Relic, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to New Relic, Follow the below steps.
<Steps>
<Step title="Get your New Relic Credentials">
<Steps>
diff --git a/docs/latest/connections/oneuptime.mdx b/docs/latest/connections/oneuptime.mdx
index aaf58ae9..e2eeb17b 100644
--- a/docs/latest/connections/oneuptime.mdx
+++ b/docs/latest/connections/oneuptime.mdx
@@ -8,7 +8,7 @@ description: 'LLM Observability with OneUptime and OpenLIT'
<img src="/images/oneuptime-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to OneUptime, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to OneUptime, Follow the below steps.
<Steps>
<Step title="Get your OneUptime OpenTelemetry Credentials">
<Steps>
diff --git a/docs/latest/connections/openobserve.mdx b/docs/latest/connections/openobserve.mdx
index 415d8bbc..9047aae7 100644
--- a/docs/latest/connections/openobserve.mdx
+++ b/docs/latest/connections/openobserve.mdx
@@ -8,7 +8,7 @@ description: 'LLM Observability with OpenObserve and OpenLIT'
<img src="/images/openobserve-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to OpenObserve, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT from your AI Application to OpenObserve, Follow the below steps.
<Steps>
<Step title="Get your OpenObserve Credentials">
<Steps>
diff --git a/docs/latest/connections/otelcol.mdx b/docs/latest/connections/otelcol.mdx
index f86f9a33..5faa9e13 100644
--- a/docs/latest/connections/otelcol.mdx
+++ b/docs/latest/connections/otelcol.mdx
@@ -3,7 +3,7 @@ title: 'OpenTelemetry Collector'
description: 'LLM Observability with OpenTelemetry Collector and OpenLIT'
---
-To send OpenTelemetry data collected by OpenLIt from your LLM Application, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
+To directly send OpenTelemetry data collected by OpenLIT SDK from your AI Application to OpenTelemetry Collector, You need to set the `otlp_endpoint` and `otlp_headers` arguments according to your needs.
```python
import openlit
diff --git a/docs/latest/connections/prometheus-jaeger.mdx b/docs/latest/connections/prometheus-jaeger.mdx
index 973ac084..ce3b0575 100644
--- a/docs/latest/connections/prometheus-jaeger.mdx
+++ b/docs/latest/connections/prometheus-jaeger.mdx
@@ -7,7 +7,7 @@ description: 'LLM Observability with Prometheus and Jaeger using OpenLIT'
<img src="/images/grafana-cloud-dashboard-1.png" />
<img src="/images/grafana-cloud-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to Prometheus and Jaeger, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to Prometheus and Jaeger, Follow the below steps.
<Steps>
<Step title="Install OpenTelemetry Collector (Optional)">
diff --git a/docs/latest/connections/prometheus-tempo.mdx b/docs/latest/connections/prometheus-tempo.mdx
index adaa6607..6fd5027a 100644
--- a/docs/latest/connections/prometheus-tempo.mdx
+++ b/docs/latest/connections/prometheus-tempo.mdx
@@ -7,7 +7,7 @@ description: 'LLM Observability with Prometheus and Grafana Tempo using OpenLIT'
<img src="/images/grafana-cloud-dashboard-1.png" />
<img src="/images/grafana-cloud-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to Prometheus and Grafana tempo, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to Prometheus and Grafana tempo, Follow the below steps.
<Steps>
<Step title="Install OpenTelemetry Collector (Optional)">
diff --git a/docs/latest/connections/signoz.mdx b/docs/latest/connections/signoz.mdx
index ee56736f..fa4a2db1 100644
--- a/docs/latest/connections/signoz.mdx
+++ b/docs/latest/connections/signoz.mdx
@@ -8,7 +8,7 @@ description: 'LLM Observability with SigNoz and OpenLIT'
<img src="/images/signoz-dashboard-2.png" />
</Frame>
-To send OpenTelemetry metrics and traces generated by OpenLIT from your LLM Application to SigNoz, Follow the below steps.
+To directly send OpenTelemetry metrics and traces generated by OpenLIT SDK from your AI Application to SigNoz, Follow the below steps.
<Steps>
<Step title="Add the following two lines to your application code:">
<Tabs>
diff --git a/docs/latest/integrations/amd-gpu.mdx b/docs/latest/integrations/amd-gpu.mdx
new file mode 100644
index 00000000..319c8916
--- /dev/null
+++ b/docs/latest/integrations/amd-gpu.mdx
@@ -0,0 +1,163 @@
+---
+title: 'Monitor AMD GPUs using OpenTelemetry'
+sidebarTitle: 'AMD GPUs'
+---
+
+<Frame>
+ <img src="/images/docs-amd-banner.gif" />
+</Frame>
+
+OpenLIT uses OpenTelemetry to help you monitor AMD GPUs. This includes tracking GPU metrics like utilization, temperature, memory usage and power consumption.
+
+
+## Get Started
+
+<CardGroup cols={2}>
+ <Card title="Using the SDK" href="#using-the-sdk" icon="file-code">
+ Collect and send GPU performance metrics directly from your application to an OpenTelemetry endpoint.
+ </Card>
+ <Card title="Using the Collector" href="#using-the-collector" icon="robot">
+ Install the OpenTelemetry GPU Collector as a Docker container to collect and send GPU performance metrics to an OpenTelemetry endpoint.
+ </Card>
+</CardGroup>
+
+
+<AccordionGroup>
+ <Accordion title="Using the SDK">
+ <Steps>
+ <Step title="Install OpenLIT">
+ Open your command line or terminal and run:
+ ```shell
+ pip install openlit
+ ```
+ </Step>
+ <Step title="Initialize OpenLIT in your Application">
+ You can set up OpenLIT in your application using either function arguments directly in your code or by using environment variables.
+ <Tabs>
+ <Tab title="Setup using function arguments">
+
+ Add the following two lines to your application code:
+
+ ```python
+ import openlit
+
+ openlit.init(
+ otlp_endpoint="YOUR_OTEL_ENDPOINT", otlp_headers="YOUR_OTEL_HEADERS"
+ collect_gpu_stats=True
+ )
+ ```
+
+ Replace:
+ 1. `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.
+ </Tab>
+ <Tab title="Setup using Environment Variables">
+
+ Add the following two lines to your application code:
+
+ ```python
+ import openlit
+
+ openlit.init(collect_gpu_stats=True)
+ ```
+
+ Then, configure the your OTLP endpoint using environment variable:
+ ```shell
+ export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_OTEL_ENDPOINT"
+ export OTEL_EXPORTER_OTLP_HEADERS= "YOUR_OTEL_HEADERS"
+ ```
+
+ Replace:
+ 1. `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.
+ </Tab>
+ </Tabs>
+ To send metrics to other Observability tools, refer to the [Connections Guide](/latest/connections/intro).
+
+ For more advanced configurations and application use cases, visit the [OpenLIT Python repository](https://github.com/openlit/openlit/tree/main/sdk/python).
+ </Step>
+ </Steps>
+ </Accordion>
+ <Accordion title="Using the Collector">
+ <Steps>
+ <Step title="Pull `otel-gpu-collector` Docker Image">
+
+ You can quickly start using the OTel GPU Collector by pulling the Docker image:
+
+ ```sh
+ docker pull ghcr.io/openlit/otel-gpu-collector:latest
+ ```
+ </Step>
+ <Step title="Run `otel-gpu-collector` Docker container">
+
+ You can quickly start using the OTel GPU Collector by pulling the Docker image:
+ Here's a quick example showing how to run the container with the required environment variables:
+
+ ```sh
+ docker run --gpus all \
+ -e GPU_APPLICATION_NAME='chatbot' \
+ -e GPU_ENVIRONMENT='staging' \
+ -e OTEL_EXPORTER_OTLP_ENDPOINT="YOUR_OTEL_ENDPOINT" \
+ -e OTEL_EXPORTER_OTLP_HEADERS="YOUR_OTEL_HEADERS" \
+ ghcr.io/openlit/otel-gpu-collector:latest
+ ```
+
+ For more advanced configurations of the collector, visit the [OTel GPU Collector repository](https://github.com/openlit/openlit/tree/main/otel-gpu-collector/).
+
+ **Note:** If you've deployed **OpenLIT** using [Docker Compose](https://github.com/openlit/openlit/blob/main/docker-compose.yml), make sure to use the host's IP address or add OTel GPU Collector to the [Docker Compose](https://github.com/openlit/openlit/blob/main/docker-compose.yml):
+
+ <AccordionGroup>
+ <Accordion title="Docker Compose: Add the following config under `services`">
+
+ ```yaml
+ otel-gpu-collector:
+ image: ghcr.io/openlit/otel-gpu-collector:latest
+ environment:
+ GPU_APPLICATION_NAME: 'chatbot'
+ GPU_ENVIRONMENT: 'staging'
+ OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4318"
+ device_requests:
+ - driver: nvidia
+ count: all
+ capabilities: [gpu]
+ depends_on:
+ - otel-collector
+ restart: always
+ ```
+
+ </Accordion>
+ <Accordion title="Host IP: Use the Host IP to connect to OTel Collector">
+
+ ```sh
+ OTEL_EXPORTER_OTLP_ENDPOINT="http://192.168.10.15:4318"
+ ```
+
+ </Accordion>
+ </AccordionGroup>
+ ### Environment Variables
+
+ OTel GPU Collector supports several environment variables for configuration. Below is a table that describes each variable:
+
+ | Environment Variable | Description | Default Value |
+ |---------------------------------|---------------------------------------------------------------|-------------------------|
+ | `GPU_APPLICATION_NAME` | Name of the application running on the GPU | `default_app` |
+ | `GPU_ENVIRONMENT` | Environment name (e.g., staging, production) | `production` |
+ | `OTEL_EXPORTER_OTLP_ENDPOINT` | OpenTelemetry OTLP endpoint URL | (required) |
+ | `OTEL_EXPORTER_OTLP_HEADERS` | Headers for authenticating with the OTLP endpoint | Ignore if using OpenLIT |
+ </Step>
+ </Steps>
+ </Accordion>
+</AccordionGroup>
+
+<Card title="Collected Metrics" href="/latest/features/metrics#gpu-metrics" icon='table'>
+Details on the types of metrics collected and their descriptions.
+</Card>
+
+---
+
+<CardGroup cols={2}>
+<Card title="Connections" href="/latest/connections/intro" icon='link'>
+Connect to your existing Observablity Stack
+</Card>
+<Card title="SDK configuration" href="/latest/sdk-configuration" icon='file-code'>
+Documentation of the configuration options for the OpenLIT SDK.
+</Card>
+</CardGroup>
\ No newline at end of file
diff --git a/docs/latest/integrations/introduction.mdx b/docs/latest/integrations/introduction.mdx
index 8fa3d101..f52dffc8 100644
--- a/docs/latest/integrations/introduction.mdx
+++ b/docs/latest/integrations/introduction.mdx
@@ -772,7 +772,7 @@ Start integrating and monitoring your applications with OpenLIT effortlessly. Ch
<CardGroup>
<Card
- title="NVIDIA GPUs"
+ title="NVIDIA"
href="/latest/integrations/nvidia-gpu"
iconType="duotone"
icon={
@@ -782,6 +782,15 @@ Start integrating and monitoring your applications with OpenLIT effortlessly. Ch
}
>
</Card>
+ <Card
+ title="AMD Radeon"
+ href="/latest/integrations/amd-gpu"
+ iconType="duotone"
+ icon={
+ <svg width="50" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 800 200"><path d="M187.888 178.12H143.52l-13.573-32.735H56.003L43.637 178.12H0L66.667 12.776h47.761zM91.155 52.285L66.912 116.53h50.913zM349.056 12.776h35.88V178.12h-41.219V74.842l-44.608 51.878h-6.301l-44.605-51.878V178.12h-41.219V12.776h35.88l53.093 61.336zM489.375 12.776c60.364 0 91.391 37.573 91.391 82.909 0 47.517-30.058 82.435-96 82.435h-68.369V12.776zm-31.762 135.039h26.906c41.457 0 53.823-28.127 53.823-52.375 0-28.368-15.276-52.363-54.308-52.363h-26.422v104.738z"/><g fill="#00a76d"><path d="M748.028 51.981h-85.259L610.797 0H800v189.21l-51.972-51.975zM662.708 137.296V62.397L609.2 115.901v74.903h74.889l53.505-53.508z"/></g></svg>
+ }
+ >
+ </Card>
</CardGroup>
---
@@ -815,6 +824,7 @@ Start integrating and monitoring your applications with OpenLIT effortlessly. Ch
| Haystack | Framework | ✔️ | ❌ |
| EmbedChain | Framework | ✔️ | ❌ |
| NVIDIA GPUs | GPU | ✔️ | ❌ |
+| AMD GPUs | GPU | ✔️ | ❌ |
---
diff --git a/docs/latest/integrations/nvidia-gpu.mdx b/docs/latest/integrations/nvidia-gpu.mdx
index 848c097b..0813c29b 100644
--- a/docs/latest/integrations/nvidia-gpu.mdx
+++ b/docs/latest/integrations/nvidia-gpu.mdx
@@ -102,6 +102,46 @@ OpenLIT uses OpenTelemetry to help you monitor NVIDIA GPUs. This includes tracki
For more advanced configurations of the collector, visit the [OTel GPU Collector repository](https://github.com/openlit/openlit/tree/main/otel-gpu-collector/).
+ **Note:** If you've deployed **OpenLIT** using [Docker Compose](https://github.com/openlit/openlit/blob/main/docker-compose.yml), make sure to use the host's IP address or add OTel GPU Collector to the [Docker Compose](https://github.com/openlit/openlit/blob/main/docker-compose.yml):
+
+ <AccordionGroup>
+ <Accordion title="Docker Compose: Add the following config under `services`">
+
+ ```yaml
+ otel-gpu-collector:
+ image: ghcr.io/openlit/otel-gpu-collector:latest
+ environment:
+ GPU_APPLICATION_NAME: 'chatbot'
+ GPU_ENVIRONMENT: 'staging'
+ OTEL_EXPORTER_OTLP_ENDPOINT: "http://otel-collector:4318"
+ device_requests:
+ - driver: nvidia
+ count: all
+ capabilities: [gpu]
+ depends_on:
+ - otel-collector
+ restart: always
+ ```
+
+ </Accordion>
+ <Accordion title="Host IP: Use the Host IP to connect to OTel Collector">
+
+ ```sh
+ OTEL_EXPORTER_OTLP_ENDPOINT="http://192.168.10.15:4318"
+ ```
+
+ </Accordion>
+ </AccordionGroup>
+ ### Environment Variables
+
+ OTel GPU Collector supports several environment variables for configuration. Below is a table that describes each variable:
+
+ | Environment Variable | Description | Default Value |
+ |---------------------------------|---------------------------------------------------------------|-------------------------|
+ | `GPU_APPLICATION_NAME` | Name of the application running on the GPU | `default_app` |
+ | `GPU_ENVIRONMENT` | Environment name (e.g., staging, production) | `production` |
+ | `OTEL_EXPORTER_OTLP_ENDPOINT` | OpenTelemetry OTLP endpoint URL | (required) |
+ | `OTEL_EXPORTER_OTLP_HEADERS` | Headers for authenticating with the OTLP endpoint | Ignore if using OpenLIT |
</Step>
</Steps>
</Accordion>
diff --git a/docs/latest/introduction.mdx b/docs/latest/introduction.mdx
index 725a5600..7fb463de 100644
--- a/docs/latest/introduction.mdx
+++ b/docs/latest/introduction.mdx
@@ -3,9 +3,11 @@ title: What is OpenLIT?
description: 'Open Source Platform for AI Engineering'
---
-<Frame>
- <img src="https://github.com/openlit/.github/raw/main/profile/assets/openlit-main-banner.png?raw=true" />
-</Frame>
+<video
+ controls
+ className="w-full aspect-video"
+ src="https://openlit.io/static/images/demo.mp4"
+></video>
**OpenLIT** allows you to simplify your AI development workflow, especially for Generative AI and LLMs. It streamlines essential tasks like experimenting with LLMs, organizing and versioning prompts, and securely handling API keys.
With just one line of code, you can enable **OpenTelemetry-native** observability, offering full-stack monitoring that includes LLMs, vector databases, and GPUs. This enables developers to confidently build AI features and applications, transitioning smoothly from testing to production.
@@ -14,6 +16,10 @@ This project proudly follows and maintains the [Semantic Conventions](https://gi
## ⚡ Features
+<Frame>
+ <img src="https://github.com/openlit/.github/raw/main/profile/assets/openlit-main-banner.png?raw=true" />
+</Frame>
+
- 📈 **Analytics Dashboard**: Monitor your AI application's health and performance with detailed dashboards that track metrics, costs, and user interactions, providing a clear view of overall efficiency.
- 🔌 **OpenTelemetry-native Observability SDKs**: Vendor-neutral SDKs to send traces and metrics to your existing observability tools.
diff --git a/docs/latest/quickstart-observability.mdx b/docs/latest/quickstart-observability.mdx
index ed583288..306233fd 100644
--- a/docs/latest/quickstart-observability.mdx
+++ b/docs/latest/quickstart-observability.mdx
@@ -266,7 +266,24 @@ flowchart TB;
</Steps>
-You're all set! Following these steps should have you on your way to effectively monitoring your LLM applications with OpenTelemetry. If you wish to send telemetry to any other backend, Refer to our [Connections](./connections/intro) guide.
+You're all set! Following these steps should have you on your way to effectively monitoring your LLM applications with OpenTelemetry.
+
+**Send Observability telemetry to other OpenTelemetry backends**
+
+```mermaid
+flowchart TB;
+ subgraph " "
+ direction LR;
+ ApplicationCode[Application Code] -->|Instrumented with| OpenLIT_SDK[OpenLIT SDK];
+ OpenLIT_SDK -->|Sends Traces & Metrics| OT_Backend[OpenTelemetry Backend];
+ end
+```
+
+If you wish to send telemetry directly from the SDK to another backend, you can stop the current Docker services by using the command below. For more details on sending the data to your existing OpenTelemetry backends, refer to our [Connections](./connections/intro) guide.
+
+```sh
+docker compose down
+```
If you have any questions or need support, reach out to our [community](https://join.slack.com/t/openlit/shared_invite/zt-2etnfttwg-TjP_7BZXfYg84oAukY8QRQ).
diff --git a/docs/mint.json b/docs/mint.json
index 54c9f186..3a90fbbc 100644
--- a/docs/mint.json
+++ b/docs/mint.json
@@ -130,7 +130,8 @@
"group": "GPUs",
"version": "latest",
"pages": [
- "latest/integrations/nvidia-gpu"
+ "latest/integrations/nvidia-gpu",
+ "latest/integrations/amd-gpu"
]
}
]