Skip to content

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

[Fix]: Grafana Cloud Documentation and Dashboard Screenshots (#210)

patcher9def0def09f48
diff --git a/docs/images/grafana-cloud-dashboard-1.png b/docs/images/grafana-cloud-dashboard-1.png
new file mode 100644
index 00000000..50175f25
Binary files /dev/null and b/docs/images/grafana-cloud-dashboard-1.png differ
diff --git a/docs/images/grafana-cloud-dashboard-2.png b/docs/images/grafana-cloud-dashboard-2.png
new file mode 100644
index 00000000..e0ae603c
Binary files /dev/null and b/docs/images/grafana-cloud-dashboard-2.png differ
diff --git a/docs/images/grafana-dashboard-1.png b/docs/images/grafana-dashboard-1.png
deleted file mode 100644
index 25d48cee..00000000
Binary files a/docs/images/grafana-dashboard-1.png and /dev/null differ
diff --git a/docs/images/grafana-dashboard-2.png b/docs/images/grafana-dashboard-2.png
deleted file mode 100644
index 5e53ed95..00000000
Binary files a/docs/images/grafana-dashboard-2.png and /dev/null differ
diff --git a/docs/images/grafana-dashboard-3.png b/docs/images/grafana-dashboard-3.png
deleted file mode 100644
index 29a9b2ba..00000000
Binary files a/docs/images/grafana-dashboard-3.png and /dev/null differ
diff --git a/docs/images/grafana-dashboard-4.png b/docs/images/grafana-dashboard-4.png
deleted file mode 100644
index 1e901a0f..00000000
Binary files a/docs/images/grafana-dashboard-4.png and /dev/null differ
diff --git a/docs/latest/connections/grafanacloud.mdx b/docs/latest/connections/grafanacloud.mdx
index 6f4a79c7..b605f9f5 100644
--- a/docs/latest/connections/grafanacloud.mdx
+++ b/docs/latest/connections/grafanacloud.mdx
@@ -4,12 +4,8 @@ description: 'LLM Observability with Grafana Cloud and OpenLIT'
---
<Frame>
- <img src="/images/grafana-dashboard-1.png" />
- <img src="/images/grafana-dashboard-2.png" />
-</Frame>
-<Frame>
- <img src="/images/grafana-dashboard-3.png" />
- <img src="/images/grafana-dashboard-4.png" />
+ <img src="/images/grafana-cloud-dashboard-1.png" />
+ <img src="/images/grafana-cloud-dashboard-2.png" />
</Frame>
To send OpenTelemetry data collected by OpenLIT from your LLM Application to Grafana Cloud, Add the snippet below to your Python application.
@@ -29,1958 +25,1909 @@ To send OpenTelemetry data collected by OpenLIT from your LLM Application to Gra
</Steps>
</Step>
<Step title="Add the following two lines to your application code:">
- <Tabs>
- <Tab title="Setup using function arguments">
- ```python
- import openlit
-
- openlit.init(
- otlp_endpoint="YOUR_GRAFANA_OTEL_GATEWAY_URL",
- otlp_headers="YOUR_GRAFANA_OTEL_GATEWAY_AUTH"
- )
- ```
-
- **Replace**:
- 1. `YOUR_GRAFANA_OTEL_GATEWAY_URL` with the `OTEL_EXPORTER_OTLP_ENDPOINT` value you copied in Step 1.
- * Example - `https://otlp-gateway-<zone>.grafana.net/otlp`
- 2. `YOUR_GRAFANA_OTEL_GATEWAY_AUTH` with the `OTEL_EXPORTER_OTLP_HEADERS` value you copied in Step 1.
- * Example - `Authorization=Basic%20<base64 encoded Instance ID and API Token>`
-
- Example Usage for monitoring `OpenAI` Usage:
-
- ```python
- from openai import OpenAI
- import openlit
-
- openlit.init(
- otlp_endpoint="https://otlp-gateway-<zone>.grafana.net/otlp",
- otlp_headers="Authorization=Basic%20<base64 encoded Instance ID and API Token>"
- )
-
- client = OpenAI(
- api_key="YOUR_OPENAI_KEY"
- )
+ <Tabs>
+ <Tab title="Setup using function arguments">
+ ```python
+ import openlit
- chat_completion = client.chat.completions.create(
- messages=[
- {
- "role": "user",
- "content": "What is LLM Observability",
- }
- ],
- model="gpt-3.5-turbo",
- )
- ```
- </Tab>
- <Tab title="Setup using Environment Variables">
- ```python
- import openlit
-
- openlit.init()
- ```
+ openlit.init(
+ otlp_endpoint="YOUR_GRAFANA_OTEL_GATEWAY_URL",
+ otlp_headers="YOUR_GRAFANA_OTEL_GATEWAY_AUTH"
+ )
+ ```
- Run the following command to configure the OTEL export endpoint:
- ```shell
- export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_GRAFANA_OTEL_GATEWAY_URL"
- export OTEL_EXPORTER_OTLP_HEADERS = "YOUR_GRAFANA_OTEL_GATEWAY_AUTH"
- ```
+ **Replace**:
+ 1. `YOUR_GRAFANA_OTEL_GATEWAY_URL` with the `OTEL_EXPORTER_OTLP_ENDPOINT` value you copied in Step 1.
+ * Example - `https://otlp-gateway-<zone>.grafana.net/otlp`
+ 2. `YOUR_GRAFANA_OTEL_GATEWAY_AUTH` with the `OTEL_EXPORTER_OTLP_HEADERS` value you copied in Step 1.
+ * Example - `Authorization=Basic%20<base64 encoded Instance ID and API Token>`
+ </Tab>
+ <Tab title="Setup using Environment Variables">
+ ```python
+ import openlit
- **Replace**:
- 1. `YOUR_GRAFANA_OTEL_GATEWAY_URL` with the `OTEL_EXPORTER_OTLP_ENDPOINT` value you copied in Step 1.
- * Example - `https://otlp-gateway-<zone>.grafana.net/otlp`
- 2. `YOUR_GRAFANA_OTEL_GATEWAY_AUTH` with the `OTEL_EXPORTER_OTLP_HEADERS` value you copied in Step 1.
- * Example - `Authorization=Basic%20<base64 encoded Instance ID and API Token>`
+ openlit.init()
+ ```
- Example Usage for monitoring `OpenAI` Usage:
+ Run the following command to configure the OTEL endpoint and headers to send metrics and traces to Grafana Cloud:
+ ```shell
+ export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_GRAFANA_OTEL_GATEWAY_URL"
+ export OTEL_EXPORTER_OTLP_HEADERS = "YOUR_GRAFANA_OTEL_GATEWAY_AUTH"
+ ```
- ```python
- from openai import OpenAI
- import openlit
-
- openlit.init()
-
- client = OpenAI(
- api_key="YOUR_OPENAI_KEY"
- )
-
- chat_completion = client.chat.completions.create(
- messages=[
+ **Replace**:
+ 1. `YOUR_GRAFANA_OTEL_GATEWAY_URL` with the `OTEL_EXPORTER_OTLP_ENDPOINT` value you copied in Step 1.
+ * Example - `https://otlp-gateway-<zone>.grafana.net/otlp`
+ 2. `YOUR_GRAFANA_OTEL_GATEWAY_AUTH` with the `OTEL_EXPORTER_OTLP_HEADERS` value you copied in Step 1.
+ * Example - `Authorization=Basic%20<base64 encoded Instance ID and API Token>`
+ </Tab>
+ </Tabs>
+ Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.
+ </Step>
+ <Step title="Import the pre-built Dashboard">
+ 1. Log into your Grafana Cloud Instance.
+ 1. Click **Dashboards** in the primary menu.
+ 1. Click **New** and select **Import** in the drop-down menu.
+ 1. Copy the dashboard JSON provided in the accordian named `Dashboard` below.
+ 1. Paste the dashboard JSON text directly into the text area.
+ 1. Click Import.
+ 1. Save the dashboard.
+ <Accordion title="Dashboard">
+ ```json
+ {
+ "annotations": {
+ "list": [
{
- "role": "user",
- "content": "What is LLM Observability",
+ "builtIn": 1,
+ "datasource": {
+ "type": "grafana",
+ "uid": "-- Grafana --"
+ },
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "type": "dashboard"
}
+ ]
+ },
+ "editable": true,
+ "fiscalYearStartMonth": 0,
+ "graphTooltip": 1,
+ "id": 12,
+ "links": [
+ {
+ "asDropdown": false,
+ "icon": "bolt",
+ "includeVars": false,
+ "keepTime": false,
+ "tags": [],
+ "targetBlank": true,
+ "title": "OpenLIT Github",
+ "tooltip": "Github",
+ "type": "link",
+ "url": "https://github.com/openlit/openlit"
+ },
+ {
+ "asDropdown": false,
+ "icon": "doc",
+ "includeVars": false,
+ "keepTime": false,
+ "tags": [],
+ "targetBlank": true,
+ "title": "OpenLIT Docs",
+ "tooltip": "Documentation",
+ "type": "link",
+ "url": "https://docs.openlit.io/"
+ }
],
- model="gpt-3.5-turbo",
- )
- ```
- </Tab>
- </Tabs>
- Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.
-</Step>
-<Step title="Import the pre-built Dashboard">
-1. Log into your Grafana Cloud Instance.
-1. Click **Dashboards** in the primary menu.
-1. Click **New** and select **Import** in the drop-down menu.
-1. Copy the dashboard JSON provided in the accordian named `Dashboard` below.
-1. Paste the dashboard JSON text directly into the text area.
-1. Click Import.
-1. Save the dashboard.
-<Accordion title="Dashboard">
- ```json
- {
- "annotations": {
- "list": [
+ "panels": [
{
- "builtIn": 1,
"datasource": {
- "type": "grafana",
- "uid": "-- Grafana --"
- },
- "enable": true,
- "hide": true,
- "iconColor": "rgba(0, 211, 255, 1)",
- "name": "Annotations & Alerts",
- "type": "dashboard"
- }
- ]
- },
- "editable": true,
- "fiscalYearStartMonth": 0,
- "graphTooltip": 1,
- "id": 12,
- "links": [
- {
- "asDropdown": false,
- "icon": "bolt",
- "includeVars": false,
- "keepTime": false,
- "tags": [],
- "targetBlank": true,
- "title": "OpenLIT Github",
- "tooltip": "Github",
- "type": "link",
- "url": "https://github.com/openlit/openlit"
- },
- {
- "asDropdown": false,
- "icon": "doc",
- "includeVars": false,
- "keepTime": false,
- "tags": [],
- "targetBlank": true,
- "title": "OpenLIT Docs",
- "tooltip": "Documentation",
- "type": "link",
- "url": "https://docs.openlit.io/"
- }
- ],
- "panels": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${DS_PROMETHEUS}"
- },
- "gridPos": {
- "h": 5,
- "w": 12,
- "x": 0,
- "y": 0
- },
- "id": 21,
- "options": {
- "code": {
- "language": "plaintext",
- "showLineNumbers": false,
- "showMiniMap": false
- },
- "content": "---\n# GenAI Observability\n\nThis dashboard displays the usage of Large Language Models (LLM) and Vector Databases, tracking OpenTelemetry Traces and Metrics sent using [OpenLIT](https://github.com/openlit/openlit).\n\n---",
- "mode": "markdown"
- },
- "pluginVersion": "11.1.0-69950",
- "transparent": true,
- "type": "text"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "grafanacloud-prom"
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "gridPos": {
+ "h": 5,
+ "w": 12,
+ "x": 0,
+ "y": 0
+ },
+ "id": 21,
+ "options": {
+ "code": {
+ "language": "plaintext",
+ "showLineNumbers": false,
+ "showMiniMap": false
+ },
+ "content": "---\n# GenAI Observability\n\nThis dashboard displays the usage of Large Language Models (LLM) and Vector Databases, tracking OpenTelemetry Traces and Metrics sent using [OpenLIT](https://github.com/openlit/openlit).\n\n---",
+ "mode": "markdown"
+ },
+ "pluginVersion": "11.1.0-69950",
+ "transparent": true,
+ "type": "text"
},
- "description": "This panel displays the LLM Request Rate ",
- "fieldConfig": {
- "defaults": {
- "color": {
- "fixedColor": "blue",
- "mode": "palette-classic-by-name"
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "red",
- "value": null
- },
- {
- "color": "#EAB839",
- "value": 10
- },
- {
- "color": "#6ED0E0",
- "value": 100
- }
- ]
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "grafanacloud-prom"
+ },
+ "description": "This panel displays the LLM Request Rate ",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "blue",
+ "mode": "palette-classic-by-name"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "red",
+ "value": null
+ },
+ {
+ "color": "#EAB839",
+ "value": 10
+ },
+ {
+ "color": "#6ED0E0",
+ "value": 100
+ }
+ ]
+ },
+ "unit": "reqps"
},
- "unit": "reqps"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 2,
- "w": 12,
- "x": 12,
- "y": 0
- },
- "id": 22,
- "options": {
- "colorMode": "background",
- "graphMode": "area",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "value_and_name",
- "wideLayout": true
+ "gridPos": {
+ "h": 2,
+ "w": 12,
+ "x": 12,
+ "y": 0
+ },
+ "id": 22,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "value_and_name",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "builder",
+ "expr": "sum(rate(gen_ai_requests_total{telemetry_sdk_name=\"openlit\"}[$__rate_interval]))",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "LLM Request Rate",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
},
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${DS_PROMETHEUS}"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "grafanacloud-prom"
+ },
+ "description": "This panel displays the VectorDB Request Rate",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "blue",
+ "mode": "palette-classic-by-name"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "red",
+ "value": null
+ },
+ {
+ "color": "#EAB839",
+ "value": 10
+ },
+ {
+ "color": "#6ED0E0",
+ "value": 100
+ }
+ ]
+ },
+ "unit": "reqps"
},
- "disableTextWrap": false,
- "editorMode": "builder",
- "expr": "sum(rate(gen_ai_requests_total{telemetry_sdk_name=\"openlit\"}[$__rate_interval]))",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "LLM Request Rate",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "grafanacloud-prom"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 12,
+ "x": 12,
+ "y": 2
+ },
+ "id": 23,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "value_and_name",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${DS_PROMETHEUS}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "builder",
+ "expr": "sum(rate(db_requests_total{telemetry_sdk_name=\"openlit\"}[$__rate_interval]))",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "VectorDB Request Rate",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
},
- "description": "This panel displays the VectorDB Request Rate",
- "fieldConfig": {
- "defaults": {
- "color": {
- "fixedColor": "blue",
- "mode": "palette-classic-by-name"
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "red",
- "value": null
- },
- {
- "color": "#EAB839",
- "value": 10
- },
- {
- "color": "#6ED0E0",
- "value": 100
- }
- ]
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the total cost incurred from using GenAI models. It reflects the financial impact of operational activities, offering insights into budgetary allocation and efficiency. Tracking this helps in effective cost management and financial planning for GenAI usage.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "blue",
+ "mode": "shades"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "currencyUSD"
},
- "unit": "reqps"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 2,
- "w": 12,
- "x": 12,
- "y": 2
- },
- "id": 23,
- "options": {
- "colorMode": "background",
- "graphMode": "area",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "value_and_name",
- "wideLayout": true
+ "gridPos": {
+ "h": 3,
+ "w": 12,
+ "x": 12,
+ "y": 4
+ },
+ "id": 2,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "auto",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum(gen_ai_usage_cost_USD_sum{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Total Usage Cost",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
},
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${DS_PROMETHEUS}"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the total number of successful requests made to the GenAI system. A successful request is one that is completed without errors, indicating seamless operation and effective utilization of the GenAI service. Tracking this helps in understanding the reliability and performance of the GenAI system.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic-by-name"
+ },
+ "mappings": [],
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
},
- "disableTextWrap": false,
- "editorMode": "builder",
- "expr": "sum(rate(db_requests_total{telemetry_sdk_name=\"openlit\"}[$__rate_interval]))",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "VectorDB Request Rate",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 5,
+ "w": 6,
+ "x": 0,
+ "y": 5
+ },
+ "id": 1,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "auto",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by() (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Total Successful GenAI Requests",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
},
- "description": "This panel displays the total cost incurred from using GenAI models. It reflects the financial impact of operational activities, offering insights into budgetary allocation and efficiency. Tracking this helps in effective cost management and financial planning for GenAI usage.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "fixedColor": "blue",
- "mode": "shades"
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
- },
- {
- "color": "red",
- "value": 80
- }
- ]
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the total count of successful VectorDB requests. It indicates requests that have been processed completely without any errors, showcasing VectorDB's effectiveness and reliability. Watching this helps gauge how well VectorDB is performing and its success in handling operations.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "thresholds"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
},
- "unit": "currencyUSD"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 3,
- "w": 12,
- "x": 12,
- "y": 4
- },
- "id": 2,
- "options": {
- "colorMode": "background",
- "graphMode": "area",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "auto",
- "wideLayout": true
+ "gridPos": {
+ "h": 5,
+ "w": 6,
+ "x": 6,
+ "y": 5
+ },
+ "id": 7,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "auto",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum(db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Total Successful VectorDB Requests",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
},
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the total number of tokens consumed by GenAI requests, providing a direct measure of usage. Monitoring this helps in assessing the demand on GenAI services and guiding resource allocation or optimization strategies.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "purple",
+ "mode": "shades"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
},
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum(gen_ai_usage_cost_USD_sum{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Total Usage Cost",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 6,
+ "x": 12,
+ "y": 7
+ },
+ "id": 3,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "area",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "auto",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum(gen_ai_usage_tokens_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Total Usage Tokens",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
},
- "description": "This panel displays the total number of successful requests made to the GenAI system. A successful request is one that is completed without errors, indicating seamless operation and effective utilization of the GenAI service. Tracking this helps in understanding the reliability and performance of the GenAI system.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic-by-name"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the average cost per use of the GenAI models and related services. It provides insights into the cost-effectiveness of interactions with GenAI, helping to identify trends in expense per operation. Monitoring this assists in optimizing budget allocation and improving cost efficiency in GenAI utilization.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "blue",
+ "mode": "shades"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "#EAB839",
+ "value": 0.5
+ },
+ {
+ "color": "red",
+ "value": 1
+ }
+ ]
+ },
+ "unit": "currencyUSD"
},
- "mappings": [],
- "min": 0,
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 3,
+ "w": 6,
+ "x": 18,
+ "y": 7
+ },
+ "id": 5,
+ "options": {
+ "colorMode": "background",
+ "graphMode": "none",
+ "justifyMode": "auto",
+ "orientation": "auto",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showPercentChange": true,
+ "textMode": "auto",
+ "wideLayout": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "avg by() (gen_ai_usage_cost_USD_sum{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Avg Usage Cost",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "transparent": true,
+ "type": "stat"
+ },
+ {
+ "datasource": {
+ "type": "tempo",
+ "uid": "${traceDatasource}"
+ },
+ "description": "This panel displays the distribution of request durations for both GenAI and VectorDB services. It highlights how long requests take to complete, from the shortest to the longest durations, offering insights into system performance and efficiency. Understanding this distribution helps in identifying bottlenecks and optimizing response times for both services.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "fixedColor": "blue",
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "fillOpacity": 81,
+ "gradientMode": "opacity",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
},
- {
- "color": "red",
- "value": 80
+ "lineWidth": 3,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
}
- ]
+ },
+ "mappings": [],
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 10
+ }
+ ]
+ },
+ "unit": "s"
},
- "unit": "none"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 6,
- "x": 0,
- "y": 5
- },
- "id": 1,
- "options": {
- "colorMode": "background",
- "graphMode": "area",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "auto",
- "wideLayout": true
- },
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by() (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Total Successful GenAI Requests",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the total count of successful VectorDB requests. It indicates requests that have been processed completely without any errors, showcasing VectorDB's effectiveness and reliability. Watching this helps gauge how well VectorDB is performing and its success in handling operations.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "thresholds"
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
+ "gridPos": {
+ "h": 7,
+ "w": 24,
+ "x": 0,
+ "y": 10
+ },
+ "id": 4,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "tempo",
+ "uid": "${traceDatasource}"
+ },
+ "filters": [
{
- "color": "green",
- "value": null
+ "id": "status",
+ "operator": "=",
+ "scope": "intrinsic",
+ "tag": "status",
+ "value": "ok",
+ "valueType": "keyword"
},
{
- "color": "red",
- "value": 80
+ "id": "f755ab99",
+ "operator": "=",
+ "scope": "span",
+ "tag": "telemetry.sdk.name",
+ "value": [
+ "openlit"
+ ],
+ "valueType": "string"
}
- ]
+ ],
+ "limit": 20,
+ "query": "{status=ok && span.telemetry.sdk.name=\"openlit\" && span.gen_ai.application_name=~\"$application\" && span.gen_ai.environment=~\"$environment\"}",
+ "queryType": "traceql",
+ "refId": "A",
+ "tableType": "traces"
+ }
+ ],
+ "title": "Request Duration Distribution",
+ "type": "histogram"
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the ranking of GenAI models based on their usage frequency. It identifies which models are most popular or in-demand, providing insights into user preferences and operational trends. Analyzing this helps in resource allocation, optimizing model availability, and understanding which GenAI features are driving usage.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-BlYlRd"
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
},
- "unit": "none"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 6,
- "x": 6,
- "y": 5
- },
- "id": 7,
- "options": {
- "colorMode": "background",
- "graphMode": "area",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "auto",
- "wideLayout": true
+ "gridPos": {
+ "h": 10,
+ "w": 14,
+ "x": 0,
+ "y": 17
+ },
+ "id": 17,
+ "options": {
+ "displayMode": "gradient",
+ "maxVizHeight": 300,
+ "minVizHeight": 16,
+ "minVizWidth": 8,
+ "namePlacement": "auto",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showUnfilled": true,
+ "sizing": "auto",
+ "valueMode": "text"
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "topk(5, sum by(gen_ai_request_model) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"}))",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "Top GenAI Models by Usage",
+ "transparent": true,
+ "type": "bargauge"
},
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the distribution of GenAI requests across different platforms such as OpenAI, Cohere, Anthropic, etc. It shows where requests are being sent, giving insights into platform popularity and usage patterns. Monitoring this helps in understanding platform preferences, and it can guide strategic decisions for integration or diversification of GenAI services.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ }
+ },
+ "mappings": [],
+ "min": 0,
+ "unit": "none"
},
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum(db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Total Successful VectorDB Requests",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 5,
+ "w": 5,
+ "x": 14,
+ "y": 17
+ },
+ "id": 16,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by(gen_ai_system) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "GenAI Requests by Platform",
+ "type": "piechart"
},
- "description": "This panel displays the total number of tokens consumed by GenAI requests, providing a direct measure of usage. Monitoring this helps in assessing the demand on GenAI services and guiding resource allocation or optimization strategies.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "fixedColor": "purple",
- "mode": "shades"
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
- },
- {
- "color": "red",
- "value": 80
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the breakdown of GenAI requests by type, such as Chat, Embedding, Image, Audio, and Fine Tuning. It reveals the diversity in usage, highlighting which types of requests are most common. Understanding this distribution assists in optimizing resources for the most demanded services and planning for future needs based on usage trends.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
}
- ]
+ },
+ "mappings": [],
+ "min": 0,
+ "unit": "none"
},
- "unit": "none"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 3,
- "w": 6,
- "x": 12,
- "y": 7
- },
- "id": 3,
- "options": {
- "colorMode": "background",
- "graphMode": "area",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "auto",
- "wideLayout": true
+ "gridPos": {
+ "h": 5,
+ "w": 5,
+ "x": 19,
+ "y": 17
+ },
+ "id": 13,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by(gen_ai_type) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "GenAI Requests by Type",
+ "type": "piechart"
},
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the distribution of GenAI requests based on the environment, such as production, development, testing, etc. It shows where the requests are being utilized, providing a clear picture of operational focus and deployment strategies. Tracking this helps in aligning resources appropriately and optimizing the performance across different environments.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ }
+ },
+ "mappings": [],
+ "min": 0,
+ "unit": "none"
},
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum(gen_ai_usage_tokens_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Total Usage Tokens",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 5,
+ "w": 5,
+ "x": 14,
+ "y": 22
+ },
+ "id": 15,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by(gen_ai_environment) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "GenAI Requests by Environment",
+ "type": "piechart"
},
- "description": "This panel displays the average cost per use of the GenAI models and related services. It provides insights into the cost-effectiveness of interactions with GenAI, helping to identify trends in expense per operation. Monitoring this assists in optimizing budget allocation and improving cost efficiency in GenAI utilization.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "fixedColor": "blue",
- "mode": "shades"
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
- },
- {
- "color": "#EAB839",
- "value": 0.5
- },
- {
- "color": "red",
- "value": 1
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the distribution of GenAI requests based on the environment, such as production, development, testing, etc. It shows where the requests are being utilized, providing a clear picture of operational focus and deployment strategies. Tracking this helps in aligning resources appropriately and optimizing the performance across different environments.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
}
- ]
+ },
+ "mappings": [],
+ "min": 0,
+ "unit": "none"
},
- "unit": "currencyUSD"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 3,
- "w": 6,
- "x": 18,
- "y": 7
- },
- "id": 5,
- "options": {
- "colorMode": "background",
- "graphMode": "none",
- "justifyMode": "auto",
- "orientation": "auto",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showPercentChange": true,
- "textMode": "auto",
- "wideLayout": true
- },
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "avg by() (gen_ai_usage_cost_USD_sum{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Avg Usage Cost",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "transparent": true,
- "type": "stat"
- },
- {
- "datasource": {
- "type": "tempo",
- "uid": "${traceDatasource}"
+ "gridPos": {
+ "h": 5,
+ "w": 5,
+ "x": 19,
+ "y": 22
+ },
+ "id": 27,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by(gen_ai_environment) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "GenAI Requests by Environment",
+ "type": "piechart"
},
- "description": "This panel displays the distribution of request durations for both GenAI and VectorDB services. It highlights how long requests take to complete, from the shortest to the longest durations, offering insights into system performance and efficiency. Understanding this distribution helps in identifying bottlenecks and optimizing response times for both services.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "fixedColor": "blue",
- "mode": "palette-classic"
- },
- "custom": {
- "fillOpacity": 81,
- "gradientMode": "opacity",
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- },
- "lineWidth": 3,
- "stacking": {
- "group": "A",
- "mode": "none"
- }
- },
- "mappings": [],
- "min": 0,
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays a comparative graph showing the average number of tokens consumed for completions and prompts against the average usage cost. It provides a visual representation of the relationship between the volume of data processed (in tokens) and the financial implications of using GenAI services. Analyzing this comparison helps in assessing cost-effectiveness and guiding strategic decisions for efficient resource utilization.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "axisBorderShow": false,
+ "axisCenteredZero": false,
+ "axisColorMode": "text",
+ "axisLabel": "",
+ "axisPlacement": "auto",
+ "barAlignment": 1,
+ "drawStyle": "line",
+ "fillOpacity": 30,
+ "gradientMode": "opacity",
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
},
- {
- "color": "red",
- "value": 10
+ "insertNulls": false,
+ "lineInterpolation": "smooth",
+ "lineStyle": {
+ "fill": "solid"
+ },
+ "lineWidth": 2,
+ "pointSize": 5,
+ "scaleDistribution": {
+ "type": "linear"
+ },
+ "showPoints": "always",
+ "spanNulls": true,
+ "stacking": {
+ "group": "A",
+ "mode": "none"
+ },
+ "thresholdsStyle": {
+ "mode": "off"
}
- ]
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
},
- "unit": "s"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 7,
- "w": 24,
- "x": 0,
- "y": 10
- },
- "id": 4,
- "options": {
- "legend": {
- "calcs": [],
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- }
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 27
+ },
+ "id": 6,
+ "options": {
+ "legend": {
+ "calcs": [],
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": true
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "avg(gen_ai_usage_prompt_tokens_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Prompt Tokens",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "builder",
+ "expr": "avg(gen_ai_usage_completion_tokens_total{telemetry_sdk_name=\"openlit\"})",
+ "fullMetaSearch": false,
+ "hide": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Completion Tokens",
+ "range": true,
+ "refId": "B",
+ "useBackend": false
+ },
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "builder",
+ "expr": "avg(gen_ai_usage_cost_USD_bucket{telemetry_sdk_name=\"openlit\"})",
+ "fullMetaSearch": false,
+ "hide": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "Usage Cost",
+ "range": true,
+ "refId": "C",
+ "useBackend": false
+ }
+ ],
+ "title": "Average Token Consumption vs. Average Usage Cost Comparison",
+ "type": "timeseries"
},
- "pluginVersion": "11.1.0-69372",
- "targets": [
- {
- "datasource": {
- "type": "tempo",
- "uid": "${traceDatasource}"
- },
- "filters": [
- {
- "id": "status",
- "operator": "=",
- "scope": "intrinsic",
- "tag": "status",
- "value": "ok",
- "valueType": "keyword"
- },
- {
- "id": "f755ab99",
- "operator": "=",
- "scope": "span",
- "tag": "telemetry.sdk.name",
- "value": [
- "openlit"
- ],
- "valueType": "string"
- }
- ],
- "limit": 20,
- "query": "{status=ok && span.telemetry.sdk.name=\"openlit\" && span.gen_ai.application_name=~\"$application\" && span.gen_ai.environment=~\"$environment\"}",
- "queryType": "traceql",
- "refId": "A",
- "tableType": "traces"
- }
- ],
- "title": "Request Duration Distribution",
- "type": "histogram"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 35
+ },
+ "id": 10,
+ "panels": [],
+ "title": "GenAI Requests",
+ "type": "row"
},
- "description": "This panel displays the ranking of GenAI models based on their usage frequency. It identifies which models are most popular or in-demand, providing insights into user preferences and operational trends. Analyzing this helps in resource allocation, optimizing model availability, and understanding which GenAI features are driving usage.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "continuous-BlYlRd"
+ {
+ "datasource": {
+ "type": "tempo",
+ "uid": "${traceDatasource}"
+ },
+ "description": "This panel displays a detailed table of GenAI request traces, providing comprehensive insights into each request's timing, source, type, and outcome. It allows for the in-depth analysis of individual requests, facilitating troubleshooting, performance monitoring, and understanding user interactions with GenAI services. Tracking this helps in identifying patterns, potential issues, and opportunities for optimization.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-BlPu"
+ },
+ "custom": {
+ "align": "auto",
+ "cellOptions": {
+ "type": "color-background"
+ },
+ "inspect": false
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ }
+ ]
+ },
+ "unit": "s"
},
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 16,
+ "w": 24,
+ "x": 0,
+ "y": 36
+ },
+ "id": 8,
+ "options": {
+ "cellHeight": "sm",
+ "footer": {
+ "countRows": false,
+ "fields": "",
+ "reducer": [
+ "sum"
+ ],
+ "show": false
+ },
+ "showHeader": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "tempo",
+ "uid": "${traceDatasource}"
+ },
+ "filters": [
{
- "color": "green",
- "value": null
+ "id": "e7e29fde",
+ "operator": "!=",
+ "scope": "span",
+ "tag": "gen_ai.type",
+ "value": [
+ "vectordb"
+ ],
+ "valueType": "string"
},
{
- "color": "red",
- "value": 80
+ "id": "status",
+ "operator": "=",
+ "scope": "intrinsic",
+ "tag": "status",
+ "value": "ok",
+ "valueType": "keyword"
}
- ]
- },
- "unit": "none"
- },
- "overrides": []
- },
- "gridPos": {
- "h": 10,
- "w": 14,
- "x": 0,
- "y": 17
+ ],
+ "limit": 20,
+ "query": "{span.gen_ai.type!=\"vectordb\" && status=ok && span.gen_ai.application_name=~\"$application\" && span.gen_ai.environment=~\"$environment\"} | select(span.gen_ai.content.prompt, span.gen_ai.content.completion, span.gen_ai.usage.cost)",
+ "queryType": "traceql",
+ "refId": "A",
+ "tableType": "traces"
+ }
+ ],
+ "transparent": true,
+ "type": "table"
},
- "id": 17,
- "options": {
- "displayMode": "gradient",
- "maxVizHeight": 300,
- "minVizHeight": 16,
- "minVizWidth": 8,
- "namePlacement": "auto",
- "orientation": "horizontal",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showUnfilled": true,
- "sizing": "auto",
- "valueMode": "text"
- },
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "topk(5, sum by(gen_ai_request_model) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"}))",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "Top GenAI Models by Usage",
- "transparent": true,
- "type": "bargauge"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the distribution of GenAI requests across different platforms such as OpenAI, Cohere, Anthropic, etc. It shows where requests are being sent, giving insights into platform popularity and usage patterns. Monitoring this helps in understanding platform preferences, and it can guide strategic decisions for integration or diversification of GenAI services.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": [],
- "min": 0,
- "unit": "none"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 5,
- "x": 14,
- "y": 17
- },
- "id": 16,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69372",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by(gen_ai_system) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "GenAI Requests by Platform",
- "type": "piechart"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the breakdown of GenAI requests by type, such as Chat, Embedding, Image, Audio, and Fine Tuning. It reveals the diversity in usage, highlighting which types of requests are most common. Understanding this distribution assists in optimizing resources for the most demanded services and planning for future needs based on usage trends.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": [],
- "min": 0,
- "unit": "none"
+ "description": "",
+ "gridPos": {
+ "h": 2,
+ "w": 24,
+ "x": 0,
+ "y": 52
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 5,
- "x": 19,
- "y": 17
- },
- "id": 13,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69372",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by(gen_ai_type) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "GenAI Requests by Type",
- "type": "piechart"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the distribution of GenAI requests based on the environment, such as production, development, testing, etc. It shows where the requests are being utilized, providing a clear picture of operational focus and deployment strategies. Tracking this helps in aligning resources appropriately and optimizing the performance across different environments.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": [],
- "min": 0,
- "unit": "none"
+ "id": 25,
+ "options": {
+ "code": {
+ "language": "plaintext",
+ "showLineNumbers": false,
+ "showMiniMap": false
+ },
+ "content": "### VectorDB\n\n---",
+ "mode": "markdown"
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 5,
- "x": 14,
- "y": 22
+ "pluginVersion": "11.1.0-69950",
+ "transparent": true,
+ "type": "text"
},
- "id": 15,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69372",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by(gen_ai_environment) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "GenAI Requests by Environment",
- "type": "piechart"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the distribution of GenAI requests based on the environment, such as production, development, testing, etc. It shows where the requests are being utilized, providing a clear picture of operational focus and deployment strategies. Tracking this helps in aligning resources appropriately and optimizing the performance across different environments.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": [],
- "min": 0,
- "unit": "none"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 5,
- "x": 19,
- "y": 22
- },
- "id": 27,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69372",
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ "description": "This panel displays the breakdown of database requests based on the type of operation, such as add, updates, deletes, and queries. It provides insights into the operational dynamics of database interactions, highlighting the most common actions performed. Understanding this distribution helps in optimizing database performance and planning for capacity based on operational needs.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "continuous-BlYlRd"
+ },
+ "mappings": [],
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "none"
},
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by(gen_ai_environment) (gen_ai_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "GenAI Requests by Environment",
- "type": "piechart"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 9,
+ "w": 14,
+ "x": 0,
+ "y": 54
+ },
+ "id": 20,
+ "options": {
+ "displayMode": "gradient",
+ "maxVizHeight": 300,
+ "minVizHeight": 16,
+ "minVizWidth": 8,
+ "namePlacement": "top",
+ "orientation": "horizontal",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "showUnfilled": true,
+ "sizing": "auto",
+ "valueMode": "text"
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by(db_operation) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "DB Requests by Operation",
+ "transparent": true,
+ "type": "bargauge"
},
- "description": "This panel displays a comparative graph showing the average number of tokens consumed for completions and prompts against the average usage cost. It provides a visual representation of the relationship between the volume of data processed (in tokens) and the financial implications of using GenAI services. Analyzing this comparison helps in assessing cost-effectiveness and guiding strategic decisions for efficient resource utilization.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "axisBorderShow": false,
- "axisCenteredZero": false,
- "axisColorMode": "text",
- "axisLabel": "",
- "axisPlacement": "auto",
- "barAlignment": 1,
- "drawStyle": "line",
- "fillOpacity": 30,
- "gradientMode": "opacity",
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- },
- "insertNulls": false,
- "lineInterpolation": "smooth",
- "lineStyle": {
- "fill": "solid"
- },
- "lineWidth": 2,
- "pointSize": 5,
- "scaleDistribution": {
- "type": "linear"
- },
- "showPoints": "always",
- "spanNulls": true,
- "stacking": {
- "group": "A",
- "mode": "none"
- },
- "thresholdsStyle": {
- "mode": "off"
- }
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
- },
- {
- "color": "red",
- "value": 80
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the distribution of requests across different VectorDB systems, such as Chroma, Pinecone, etc. It highlights which VectorDB services are most frequently used, providing insights into system preference and usage patterns. Monitoring this helps in evaluating the performance and scalability of each VectorDB system, guiding strategic decisions for technology adoption and integration.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
}
- ]
+ },
+ "mappings": []
},
- "unit": "none"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 8,
- "w": 24,
- "x": 0,
- "y": 27
- },
- "id": 6,
- "options": {
- "legend": {
- "calcs": [],
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": true
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "targets": [
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "avg(gen_ai_usage_prompt_tokens_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Prompt Tokens",
- "range": true,
- "refId": "A",
- "useBackend": false
+ "gridPos": {
+ "h": 5,
+ "w": 10,
+ "x": 14,
+ "y": 54
},
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "builder",
- "expr": "avg(gen_ai_usage_completion_tokens_total{telemetry_sdk_name=\"openlit\"})",
- "fullMetaSearch": false,
- "hide": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Completion Tokens",
- "range": true,
- "refId": "B",
- "useBackend": false
+ "id": 18,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": true
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
},
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "builder",
- "expr": "avg(gen_ai_usage_cost_USD_bucket{telemetry_sdk_name=\"openlit\"})",
- "fullMetaSearch": false,
- "hide": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "Usage Cost",
- "range": true,
- "refId": "C",
- "useBackend": false
- }
- ],
- "title": "Average Token Consumption vs. Average Usage Cost Comparison",
- "type": "timeseries"
- },
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 35
- },
- "id": 10,
- "panels": [],
- "title": "GenAI Requests",
- "type": "row"
- },
- {
- "datasource": {
- "type": "tempo",
- "uid": "${traceDatasource}"
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "code",
+ "expr": "sum by(db_system) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "DB Requests by System",
+ "type": "piechart"
},
- "description": "This panel displays a detailed table of GenAI request traces, providing comprehensive insights into each request's timing, source, type, and outcome. It allows for the in-depth analysis of individual requests, facilitating troubleshooting, performance monitoring, and understanding user interactions with GenAI services. Tracking this helps in identifying patterns, potential issues, and opportunities for optimization.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "continuous-BlPu"
- },
- "custom": {
- "align": "auto",
- "cellOptions": {
- "type": "color-background"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the breakdown of DB requests by application name, offering insights into which applications are leveraging VectorDB services the most. This identification helps in understanding application-specific demand, guiding resource allocation, and supporting targeted optimization efforts for enhanced efficiency in application performance.\"",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
},
- "inspect": false
- },
- "mappings": [],
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
}
- ]
+ },
+ "mappings": [],
+ "min": 0,
+ "unit": "none"
},
- "unit": "s"
+ "overrides": []
},
- "overrides": []
- },
- "gridPos": {
- "h": 16,
- "w": 24,
- "x": 0,
- "y": 36
- },
- "id": 8,
- "options": {
- "cellHeight": "sm",
- "footer": {
- "countRows": false,
- "fields": "",
- "reducer": [
- "sum"
- ],
- "show": false
- },
- "showHeader": true
+ "gridPos": {
+ "h": 4,
+ "w": 5,
+ "x": 14,
+ "y": 59
+ },
+ "id": 19,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "builder",
+ "expr": "sum by(gen_ai_application_name) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "DB Requests by Application",
+ "type": "piechart"
},
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "tempo",
- "uid": "${traceDatasource}"
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "description": "This panel displays the breakdown of DB requests by application name, offering insights into which applications are leveraging VectorDB services the most. This identification helps in understanding application-specific demand, guiding resource allocation, and supporting targeted optimization efforts for enhanced efficiency in application performance.\"",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic"
+ },
+ "custom": {
+ "hideFrom": {
+ "legend": false,
+ "tooltip": false,
+ "viz": false
+ }
+ },
+ "mappings": [],
+ "min": 0,
+ "unit": "none"
},
- "filters": [
- {
- "id": "e7e29fde",
- "operator": "!=",
- "scope": "span",
- "tag": "gen_ai.type",
- "value": [
- "vectordb"
- ],
- "valueType": "string"
- },
- {
- "id": "status",
- "operator": "=",
- "scope": "intrinsic",
- "tag": "status",
- "value": "ok",
- "valueType": "keyword"
- }
- ],
- "limit": 20,
- "query": "{span.gen_ai.type!=\"vectordb\" && status=ok && span.gen_ai.application_name=~\"$application\" && span.gen_ai.environment=~\"$environment\"} | select(span.gen_ai.content.prompt, span.gen_ai.content.completion, span.gen_ai.usage.cost)",
- "queryType": "traceql",
- "refId": "A",
- "tableType": "traces"
- }
- ],
- "transparent": true,
- "type": "table"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "",
- "gridPos": {
- "h": 2,
- "w": 24,
- "x": 0,
- "y": 52
- },
- "id": 25,
- "options": {
- "code": {
- "language": "plaintext",
- "showLineNumbers": false,
- "showMiniMap": false
- },
- "content": "### VectorDB\n\n---",
- "mode": "markdown"
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 4,
+ "w": 5,
+ "x": 19,
+ "y": 59
+ },
+ "id": 26,
+ "options": {
+ "legend": {
+ "displayMode": "list",
+ "placement": "bottom",
+ "showLegend": false
+ },
+ "pieType": "pie",
+ "reduceOptions": {
+ "calcs": [
+ "lastNotNull"
+ ],
+ "fields": "",
+ "values": false
+ },
+ "tooltip": {
+ "mode": "single",
+ "sort": "none"
+ }
+ },
+ "pluginVersion": "11.1.0-69372",
+ "targets": [
+ {
+ "datasource": {
+ "type": "prometheus",
+ "uid": "${metricsDatasource}"
+ },
+ "disableTextWrap": false,
+ "editorMode": "builder",
+ "expr": "sum by(gen_ai_application_name) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
+ "fullMetaSearch": false,
+ "includeNullMetadata": true,
+ "instant": false,
+ "legendFormat": "__auto",
+ "range": true,
+ "refId": "A",
+ "useBackend": false
+ }
+ ],
+ "title": "DB Requests by Application",
+ "type": "piechart"
},
- "pluginVersion": "11.1.0-69950",
- "transparent": true,
- "type": "text"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
+ {
+ "collapsed": false,
+ "gridPos": {
+ "h": 1,
+ "w": 24,
+ "x": 0,
+ "y": 63
+ },
+ "id": 12,
+ "panels": [],
+ "title": "VectorDB Requests",
+ "type": "row"
},
- "description": "This panel displays the breakdown of database requests based on the type of operation, such as add, updates, deletes, and queries. It provides insights into the operational dynamics of database interactions, highlighting the most common actions performed. Understanding this distribution helps in optimizing database performance and planning for capacity based on operational needs.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "continuous-BlYlRd"
+ {
+ "datasource": {
+ "type": "tempo",
+ "uid": "${traceDatasource}"
+ },
+ "description": "This panel displays a table with detailed traces of VectorDB requests, offering insights into specifics like request timing, source, operation type, and results. It's designed for thorough examination of VectorDB interactions, enabling precise troubleshooting, performance evaluation, and user behavior understanding. By monitoring these traces, users can spot trends, diagnose problems, and improve operational efficiency in handling VectorDB requests.",
+ "fieldConfig": {
+ "defaults": {
+ "color": {
+ "mode": "palette-classic-by-name"
+ },
+ "custom": {
+ "align": "auto",
+ "cellOptions": {
+ "type": "color-background"
+ },
+ "inspect": false
+ },
+ "mappings": [],
+ "min": 0,
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ },
+ "unit": "s"
},
- "mappings": [],
- "min": 0,
- "thresholds": {
- "mode": "absolute",
- "steps": [
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 15,
+ "w": 24,
+ "x": 0,
+ "y": 64
+ },
+ "id": 9,
+ "options": {
+ "cellHeight": "sm",
+ "footer": {
+ "countRows": false,
+ "fields": "",
+ "reducer": [
+ "sum"
+ ],
+ "show": false
+ },
+ "showHeader": true
+ },
+ "pluginVersion": "11.1.0-69950",
+ "targets": [
+ {
+ "datasource": {
+ "type": "tempo",
+ "uid": "${traceDatasource}"
+ },
+ "filters": [
{
- "color": "green",
- "value": null
+ "id": "e7e29fde",
+ "operator": "=",
+ "scope": "span",
+ "tag": "gen_ai.type",
+ "value": [
+ "vectordb"
+ ],
+ "valueType": "string"
},
{
- "color": "red",
- "value": 80
+ "id": "status",
+ "operator": "=",
+ "scope": "intrinsic",
+ "tag": "status",
+ "value": "ok",
+ "valueType": "keyword"
}
- ]
- },
- "unit": "none"
- },
- "overrides": []
- },
- "gridPos": {
- "h": 9,
- "w": 14,
- "x": 0,
- "y": 54
- },
- "id": 20,
- "options": {
- "displayMode": "gradient",
- "maxVizHeight": 300,
- "minVizHeight": 16,
- "minVizWidth": 8,
- "namePlacement": "top",
- "orientation": "horizontal",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "showUnfilled": true,
- "sizing": "auto",
- "valueMode": "text"
- },
- "pluginVersion": "11.1.0-69950",
- "targets": [
+ ],
+ "limit": 20,
+ "query": "{span.gen_ai.type=\"vectordb\" && status=ok && span.gen_ai.application_name=~\"$application\" && span.gen_ai.environment=~\"$environment\"}",
+ "queryType": "traceql",
+ "refId": "A",
+ "tableType": "traces"
+ }
+ ],
+ "transparent": true,
+ "type": "table"
+ }
+ ],
+ "refresh": "",
+ "schemaVersion": 39,
+ "tags": [
+ "LLM",
+ "VectorDB",
+ "GenAI"
+ ],
+ "templating": {
+ "list": [
{
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by(db_operation) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "DB Requests by Operation",
- "transparent": true,
- "type": "bargauge"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the distribution of requests across different VectorDB systems, such as Chroma, Pinecone, etc. It highlights which VectorDB services are most frequently used, providing insights into system preference and usage patterns. Monitoring this helps in evaluating the performance and scalability of each VectorDB system, guiding strategic decisions for technology adoption and integration.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": []
+ "hide": 0,
+ "includeAll": false,
+ "label": "Trace Datasource",
+ "multi": false,
+ "name": "traceDatasource",
+ "options": [],
+ "query": "tempo",
+ "queryValue": "",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "type": "datasource"
},
- "overrides": []
- },
- "gridPos": {
- "h": 5,
- "w": 10,
- "x": 14,
- "y": 54
- },
- "id": 18,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": true
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69950",
- "targets": [
{
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "code",
- "expr": "sum by(db_system) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "DB Requests by System",
- "type": "piechart"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the breakdown of DB requests by application name, offering insights into which applications are leveraging VectorDB services the most. This identification helps in understanding application-specific demand, guiding resource allocation, and supporting targeted optimization efforts for enhanced efficiency in application performance.\"",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": [],
- "min": 0,
- "unit": "none"
+ "hide": 0,
+ "includeAll": false,
+ "label": "Metrics Datasource",
+ "multi": false,
+ "name": "metricsDatasource",
+ "options": [],
+ "query": "prometheus",
+ "queryValue": "",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "type": "datasource"
},
- "overrides": []
- },
- "gridPos": {
- "h": 4,
- "w": 5,
- "x": 14,
- "y": 59
- },
- "id": 19,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69372",
- "targets": [
{
+ "current": {
+ "selected": true,
+ "text": [
+ "All"
+ ],
+ "value": [
+ "$__all"
+ ]
+ },
"datasource": {
"type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "builder",
- "expr": "sum by(gen_ai_application_name) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
- }
- ],
- "title": "DB Requests by Application",
- "type": "piechart"
- },
- {
- "datasource": {
- "type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "description": "This panel displays the breakdown of DB requests by application name, offering insights into which applications are leveraging VectorDB services the most. This identification helps in understanding application-specific demand, guiding resource allocation, and supporting targeted optimization efforts for enhanced efficiency in application performance.\"",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic"
- },
- "custom": {
- "hideFrom": {
- "legend": false,
- "tooltip": false,
- "viz": false
- }
- },
- "mappings": [],
- "min": 0,
- "unit": "none"
+ "uid": "grafanacloud-prom"
+ },
+ "definition": "label_values(gen_ai_application_name)",
+ "hide": 0,
+ "includeAll": true,
+ "label": "Application",
+ "multi": true,
+ "name": "application",
+ "options": [],
+ "query": {
+ "qryType": 1,
+ "query": "label_values(gen_ai_application_name)",
+ "refId": "PrometheusVariableQueryEditor-VariableQuery"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "type": "query"
},
- "overrides": []
- },
- "gridPos": {
- "h": 4,
- "w": 5,
- "x": 19,
- "y": 59
- },
- "id": 26,
- "options": {
- "legend": {
- "displayMode": "list",
- "placement": "bottom",
- "showLegend": false
- },
- "pieType": "pie",
- "reduceOptions": {
- "calcs": [
- "lastNotNull"
- ],
- "fields": "",
- "values": false
- },
- "tooltip": {
- "mode": "single",
- "sort": "none"
- }
- },
- "pluginVersion": "11.1.0-69372",
- "targets": [
{
+ "current": {
+ "selected": true,
+ "text": [
+ "All"
+ ],
+ "value": [
+ "$__all"
+ ]
+ },
"datasource": {
"type": "prometheus",
- "uid": "${metricsDatasource}"
- },
- "disableTextWrap": false,
- "editorMode": "builder",
- "expr": "sum by(gen_ai_application_name) (db_requests_total{telemetry_sdk_name=\"openlit\", gen_ai_application_name=~\"$application\", gen_ai_environment=~\"$environment\"})",
- "fullMetaSearch": false,
- "includeNullMetadata": true,
- "instant": false,
- "legendFormat": "__auto",
- "range": true,
- "refId": "A",
- "useBackend": false
+ "uid": "grafanacloud-prom"
+ },
+ "definition": "label_values(gen_ai_environment)",
+ "hide": 0,
+ "includeAll": true,
+ "label": "Environment",
+ "multi": true,
+ "name": "environment",
+ "options": [],
+ "query": {
+ "qryType": 1,
+ "query": "label_values(gen_ai_environment)",
+ "refId": "PrometheusVariableQueryEditor-VariableQuery"
+ },
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "type": "query"
}
- ],
- "title": "DB Requests by Application",
- "type": "piechart"
+ ]
},
- {
- "collapsed": false,
- "gridPos": {
- "h": 1,
- "w": 24,
- "x": 0,
- "y": 63
- },
- "id": 12,
- "panels": [],
- "title": "VectorDB Requests",
- "type": "row"
+ "time": {
+ "from": "now-24h",
+ "to": "now"
},
- {
- "datasource": {
- "type": "tempo",
- "uid": "${traceDatasource}"
- },
- "description": "This panel displays a table with detailed traces of VectorDB requests, offering insights into specifics like request timing, source, operation type, and results. It's designed for thorough examination of VectorDB interactions, enabling precise troubleshooting, performance evaluation, and user behavior understanding. By monitoring these traces, users can spot trends, diagnose problems, and improve operational efficiency in handling VectorDB requests.",
- "fieldConfig": {
- "defaults": {
- "color": {
- "mode": "palette-classic-by-name"
- },
- "custom": {
- "align": "auto",
- "cellOptions": {
- "type": "color-background"
- },
- "inspect": false
- },
- "mappings": [],
- "min": 0,
- "thresholds": {
- "mode": "absolute",
- "steps": [
- {
- "color": "green",
- "value": null
- },
- {
- "color": "red",
- "value": 80
- }
- ]
- },
- "unit": "s"
- },
- "overrides": []
- },
- "gridPos": {
- "h": 15,
- "w": 24,
- "x": 0,
- "y": 64
- },
- "id": 9,
- "options": {
- "cellHeight": "sm",
- "footer": {
- "countRows": false,
- "fields": "",
- "reducer": [
- "sum"
- ],
- "show": false
- },
- "showHeader": true
- },
- "pluginVersion": "11.1.0-69950",
- "targets": [
- {
- "datasource": {
- "type": "tempo",
- "uid": "${traceDatasource}"
- },
- "filters": [
- {
- "id": "e7e29fde",
- "operator": "=",
- "scope": "span",
- "tag": "gen_ai.type",
- "value": [
- "vectordb"
- ],
- "valueType": "string"
- },
- {
- "id": "status",
- "operator": "=",
- "scope": "intrinsic",
- "tag": "status",
- "value": "ok",
- "valueType": "keyword"
- }
- ],
- "limit": 20,
- "query": "{span.gen_ai.type=\"vectordb\" && status=ok && span.gen_ai.application_name=~\"$application\" && span.gen_ai.environment=~\"$environment\"}",
- "queryType": "traceql",
- "refId": "A",
- "tableType": "traces"
- }
- ],
- "transparent": true,
- "type": "table"
- }
- ],
- "refresh": "",
- "schemaVersion": 39,
- "tags": [
- "LLM",
- "VectorDB",
- "GenAI"
- ],
- "templating": {
- "list": [
- {
- "hide": 0,
- "includeAll": false,
- "label": "Trace Datasource",
- "multi": false,
- "name": "traceDatasource",
- "options": [],
- "query": "tempo",
- "queryValue": "",
- "refresh": 1,
- "regex": "",
- "skipUrlSync": false,
- "type": "datasource"
- },
- {
- "hide": 0,
- "includeAll": false,
- "label": "Metrics Datasource",
- "multi": false,
- "name": "metricsDatasource",
- "options": [],
- "query": "prometheus",
- "queryValue": "",
- "refresh": 1,
- "regex": "",
- "skipUrlSync": false,
- "type": "datasource"
- },
- {
- "current": {
- "selected": true,
- "text": [
- "All"
- ],
- "value": [
- "$__all"
- ]
- },
- "datasource": {
- "type": "prometheus",
- "uid": "grafanacloud-prom"
- },
- "definition": "label_values(gen_ai_application_name)",
- "hide": 0,
- "includeAll": true,
- "label": "Application",
- "multi": true,
- "name": "application",
- "options": [],
- "query": {
- "qryType": 1,
- "query": "label_values(gen_ai_application_name)",
- "refId": "PrometheusVariableQueryEditor-VariableQuery"
- },
- "refresh": 1,
- "regex": "",
- "skipUrlSync": false,
- "sort": 0,
- "type": "query"
- },
- {
- "current": {
- "selected": true,
- "text": [
- "All"
- ],
- "value": [
- "$__all"
- ]
- },
- "datasource": {
- "type": "prometheus",
- "uid": "grafanacloud-prom"
- },
- "definition": "label_values(gen_ai_environment)",
- "hide": 0,
- "includeAll": true,
- "label": "Environment",
- "multi": true,
- "name": "environment",
- "options": [],
- "query": {
- "qryType": 1,
- "query": "label_values(gen_ai_environment)",
- "refId": "PrometheusVariableQueryEditor-VariableQuery"
- },
- "refresh": 1,
- "regex": "",
- "skipUrlSync": false,
- "sort": 1,
- "type": "query"
- }
- ]
- },
- "time": {
- "from": "now-24h",
- "to": "now"
- },
- "timeRangeUpdatedDuringEditOrView": false,
- "timepicker": {},
- "timezone": "browser",
- "title": "GenAI Observability",
- "uid": "cdiz9piuoa3gge",
- "version": 15,
- "weekStart": ""
- }
- ```
-</Accordion>
+ "timeRangeUpdatedDuringEditOrView": false,
+ "timepicker": {},
+ "timezone": "browser",
+ "title": "GenAI Observability",
+ "uid": "cdiz9piuoa3gge",
+ "version": 15,
+ "weekStart": ""
+ }
+ ```
+ </Accordion>
-</Step>
+ </Step>
</Steps>