Skip to content

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

15.3 KB
Raw Blame
---
title: 'Configuration'
description: 'Configure the OpenLIT SDK for OpenTelemetry-native LLM observability, cost tracking, and performance monitoring'
icon: 'sliders'
---

Configure OpenLIT SDK for AI monitoring and model performance tracking using flexible instrumentation methods. Choose from Manual instrumentation or Zero-code instrumentation for complete LLM observability:

<Tabs>
<Tab title="Python">

### Manual instrumentation (SDK)

```python
import openlit

openlit.init(
    service_name="my-ai-app",
    environment="production",
    otlp_endpoint="https://otel-endpoint.com"
)
```

### Zero-code instrumentation (CLI)

```bash
export OTEL_SERVICE_NAME=my-ai-app
export OTEL_DEPLOYMENT_ENVIRONMENT=production
openlit-instrument python your_app.py
```

### Configuration parameters

Customize OpenLIT SDK behavior for your specific instrumentation needs:

| Parameter | CLI Argument | Environment Variable | Description | Default | Required |
|-----------|--------------|---------------------|-------------|---------|----------|
| `environment` | `--environment` | `OTEL_DEPLOYMENT_ENVIRONMENT` | Deployment environment | `"default"` | No |
| `service_name` | `--service_name` | `OTEL_SERVICE_NAME` | Service name for tracing | `"default"` | No |
| `otlp_endpoint` | `--otlp_endpoint` | `OTEL_EXPORTER_OTLP_ENDPOINT` | OpenTelemetry endpoint for LLM monitoring data export | `None` | No |
| `otlp_headers` | `--otlp_headers` | `OTEL_EXPORTER_OTLP_HEADERS` | Authentication headers for enterprise monitoring backends | `None` | No |
| `disable_batch` | `--disable_batch` | `OPENLIT_DISABLE_BATCH` | Disable batch span processing | `False` | No |
| `capture_message_content` | `--capture_message_content` | `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | Enable LLM prompt and response content tracing for debugging | `True` | No |
| `disabled_instrumentors` | `--disabled_instrumentors` | `OPENLIT_DISABLED_INSTRUMENTORS` | Disable specific AI service instrumentations (comma-separated) | `None` | No |
| `disable_metrics` | `--disable_metrics` | `OPENLIT_DISABLE_METRICS` | Disable cost tracking and performance metrics collection | `False` | No |
| `max_content_length` | `--max_content_length` | `OPENLIT_MAX_CONTENT_LENGTH` | Maximum character length for captured prompt/completion content. `None` (default) means no truncation. Set to a positive integer to truncate content exceeding that length. `0` or `-1` also disable truncation. | `None` | No |
| `pricing_json` | `--pricing_json` | `OPENLIT_PRICING_JSON` | Custom pricing configuration for accurate LLM cost tracking | `None` | No |
| `disable_events` | `--disable_events` | `OPENLIT_DISABLE_EVENTS` | Disable OTel Logger event emission | `False` | No |
| `collect_system_metrics` | `--collect_system_metrics` | `OPENLIT_COLLECT_SYSTEM_METRICS` | Comprehensive system monitoring (CPU, memory, disk, network, GPU) for AI workloads | `False` | No |
| `custom_metrics_attributes` | N/A | N/A | Custom key-value attributes applied to every metric recording. Useful for grouping metrics by custom tags (e.g., client ID, team, project). Reserved keys like `service.name` cannot be overwritten. | `None` | No |

### Database instrumentation options

These options apply to database instrumentations like PostgreSQL (psycopg3):

| Parameter | CLI Argument | Environment Variable | Description | Default | Required |
|-----------|--------------|---------------------|-------------|---------|----------|
| `capture_db_parameters` | `--capture_db_parameters` | `OPENLIT_CAPTURE_DB_PARAMETERS` | Capture database query parameters in OTel per-key format (`db.query.parameter.<key>`) | `False` | No |

### Evaluation export options

Configure how evaluation results are exported:

<Warning>
**Security Notice**: Enabling `capture_db_parameters` records query parameters (the values passed to `$1`, `$2`, etc.) in your traces using the OTel per-key format (`db.query.parameter.<key>`). This may expose sensitive data like passwords, API keys, or personal information. Only enable in development environments or when you're certain parameters don't contain sensitive data.
</Warning>

### Deprecated parameters

| Parameter | CLI Argument | Environment Variable | Description | Default | Required |
|-----------|--------------|---------------------|-------------|---------|----------|
| `application_name` | `--application_name` | `OTEL_SERVICE_NAME` | Application name for tracing (**deprecated**, use `service_name`) | `"default"` | No |
| `collect_gpu_stats` | `--collect_gpu_stats` | `OPENLIT_COLLECT_GPU_STATS` | Enable GPU statistics collection (**deprecated**, use `collect_system_metrics`) | `False` | No |

<Info>Environment variables take precedence over CLI arguments, which take precedence over SDK parameters.</Info>

### Resource attributes

Additional resource attributes can be controlled using standard OpenTelemetry environment variables for enhanced metadata and observability context:

| Environment Variable | Description | Example |
|---------------------|-------------|---------|
| `OTEL_RESOURCE_ATTRIBUTES` | Key-value pairs for resource attributes | `service.version=1.0.0,deployment.environment=production` |
| `OTEL_SERVICE_VERSION` | Version of the service | `1.2.3` |
| `OTEL_RESOURCE_ATTRIBUTES_POD_NAME` | Kubernetes pod name (if applicable) | `my-ai-app-pod-xyz` |
| `OTEL_RESOURCE_ATTRIBUTES_NODE_NAME` | Kubernetes node name (if applicable) | `node-123` |

**Example:**
```bash
# Set resource attributes for better trace organization
export OTEL_RESOURCE_ATTRIBUTES="service.version=2.1.0,team=ai-platform,cost.center=engineering"
export OTEL_SERVICE_VERSION=2.1.0

# Run with enhanced metadata
openlit-instrument python your_ai_app.py
```

These attributes enhance trace metadata for better filtering, grouping, and analysis in your observability platform.

## Prompt Hub - `openlit.get_prompt()`

Advanced prompt management and version control for production LLM applications. Configure OpenLIT Prompt Hub for centralized prompt governance and tracking:

| Parameter         | Description                                                                                                                        |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------|
| `url`             | Sets the OpenLIT URL. Defaults to the `OPENLIT_URL` environment variable.                                                          |
| `api_key`         | Sets the OpenLIT API Key. Can also be provided via the `OPENLIT_API_KEY` environment variable.                                     |
| `name`            | Unique prompt identifier for retrieval. Use with `prompt_id` for specific prompt versioning                                        |
| `prompt_id`       | Numeric ID for direct prompt access. Enables precise prompt version control. Optional                                              |
| `version`         | Specific prompt version retrieval for consistent AI behavior across deployments. Optional                                          |
| `shouldCompile`   | Enable dynamic prompt compilation with variables for personalized LLM interactions. Optional                                       |
| `variables`       | Dynamic variables for prompt template compilation and customization. Optional                                                      |
| `meta_properties` | Tracking metadata for prompt usage analytics and audit trails in production. Optional                                              |

## Vault - `openlit.get_secrets()`

Enterprise-grade secret management for AI applications. Configure OpenLIT Vault for secure API key and credential handling in production LLM deployments:

| Parameter         | Description                                                                                                                        |
|-------------------|------------------------------------------------------------------------------------------------------------------------------------|
| `url`             | Sets the Openlit URL. Defaults to the `OPENLIT_URL` environment variable.                                                          |
| `api_key`         | Sets the OpenLIT API Key. Can also be provided via the `OPENLIT_API_KEY` environment variable.                                     |
| `key`            | Specific secret key retrieval for individual credential access. Optional                                                           |
| `should_set_env` | Automatically set retrieved secrets as environment variables for seamless application integration. Optional                        |
| `tags`           | Tag-based secret filtering for organized credential management across different AI services. Optional                               |

</Tab>
<Tab title="Typescript">

## Configuration parameters

Customize OpenLIT SDK behavior for your specific instrumentation needs:

| Argument                 | Description                                                                          | Default Value                                                      | Required |
| ------------------------ | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------ | -------- |
| `environment`            | The deployment environment of the application.                                       | `"default"`                                                        | No      |
| `applicationName`       | Identifies the name of your application.                                             | `"default"`                                                        | No      |
| `tracer`                 | An instance of OpenTelemetry Tracer for tracing operations.                          | `undefined`                                                             | No       |
| `otlpEndpoint`          | OpenTelemetry endpoint for LLM monitoring data transmission to enterprise backends.  | `undefined`                                                             | No       |
| `otlpHeaders`           | Authentication headers for secure integration with monitoring platforms like Datadog, Grafana. | `undefined`                                                             | No       |
| `disableBatch`          | A flag to disable batch span processing, favoring immediate dispatch.                | `true`                                                            | No       |
| `traceContent`          | Enable LLM prompt and response content capture for comprehensive debugging.          | `true`                                                             | No       |
| `disabledInstrumentations` | Disable specific AI service instrumentations for customized monitoring scope.       | `undefined`                                                             | No       |
| `instrumentations`        | Object of instrumentation modules for manual patching                                          | `undefined`                                                            | No       |
| `pricing_json`           | Custom pricing configuration URL for accurate LLM cost tracking and optimization.    | `https://github.com/openlit/openlit/blob/main/assets/pricing.json` | No       |

### Resource attributes

Additional resource attributes can be controlled using standard OpenTelemetry environment variables for enhanced metadata and observability context:

| Environment Variable | Description | Example |
|---------------------|-------------|---------|
| `OTEL_RESOURCE_ATTRIBUTES` | Key-value pairs for resource attributes | `service.version=1.0.0,deployment.environment=production` |
| `OTEL_SERVICE_VERSION` | Version of the service | `1.2.3` |
| `OTEL_RESOURCE_ATTRIBUTES_POD_NAME` | Kubernetes pod name (if applicable) | `my-ai-app-pod-xyz` |
| `OTEL_RESOURCE_ATTRIBUTES_NODE_NAME` | Kubernetes node name (if applicable) | `node-123` |

These attributes enhance trace metadata for better filtering, grouping, and analysis in your observability platform.

## Prompt Hub - `Openlit.getPrompt()`

Advanced prompt management for Node.js AI applications. Configure centralized prompt governance and version control for production LLM deployments:

| Parameter        | Description                                                                                                                        |
|------------------|------------------------------------------------------------------------------------------------------------------------------------|
| `url`            | Sets the OpenLIT URL. Defaults to the `OPENLIT_URL` environment variable or `http://127.0.0.1:3000` if not set.                    |
| `apiKey`         | Sets the OpenLIT API Key. Can also be provided via the `OPENLIT_API_KEY` environment variable.                                     |
| `name`           | Unique prompt identifier for consistent retrieval across deployments.                                                              |
| `promptId`       | Numeric prompt ID for direct access and version control. Optional                                                                  |
| `version`        | Specific prompt version for consistent AI behavior in production. Optional                                                          | 
| `shouldCompile`  | Enable dynamic prompt compilation with variables for personalized interactions. Optional                                           |
| `variables`      | Dynamic template variables for prompt customization and personalization. Optional                                                  | 
| `metaProperties` | Usage tracking metadata for prompt analytics and audit trails. Optional                                                            | 


## Vault - `Openlit.getSecrets()`

Enterprise-grade secret management for Node.js AI applications. Configure secure credential handling for production LLM deployments:

| Parameter        | Description                                                                                                  |
|------------------|------------------------------------------------------------------------------------------------------------------------------------|
| `url`            | Sets the Openlit URL. Defaults to the `OPENLIT_URL` environment variable or `http://127.0.0.1:3000` if not set.                    |
| `apiKey`         | Sets the OpenLIT API Key. Can also be provided via the `OPENLIT_API_KEY` environment variable.                                     |
| `key`            | Specific secret key for individual credential retrieval. Optional                                                  |
| `tags`           | Tag-based secret filtering for organized credential management across AI services. Optional                         |
| `shouldSetEnv`   | Automatically set secrets as environment variables for seamless application integration. Optional                   |

</Tab>
</Tabs>

---

<CardGroup cols={3}>
  <Card title="Deploy OpenLIT" href="/latest/openlit/installation" icon='circle-down'>
    Deployment options for scalable LLM monitoring infrastructure
  </Card>
  <Card title="Integrations" href="/latest/sdk/integrations/overview" icon='circle-nodes'>
    60+ AI integrations with automatic instrumentation and performance tracking
  </Card>
  <Card title="Destinations" href="/latest/sdk/destinations/overview" icon='link'>
    Send elemetry to Datadog, Grafana, New Relic, and other observability stacks
  </Card>
</CardGroup>