Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
OpenLIT Go SDK
OpenTelemetry-native SDK for OpenLIT, the open-source agent harness engineering platform. Instrument LLM applications in Go to trace, evaluate, and improve the harness around your models — with automatic instrumentation for popular LLM providers.
Features
- 🚀 One-line initialization - Get started with minimal code
- 📊 OpenTelemetry-native - Built on OpenTelemetry standards
- 💰 Cost tracking - Automatic cost calculation for LLM requests
- 📈 Metrics collection - Token usage, latency, and performance metrics
- 🔄 Streaming support - Full support for streaming responses
- 🔌 Flexible integration - Works with multiple Go client libraries
Supported Integrations
- ✅ OpenAI - Chat completions, embeddings, images, audio
- ✅ Anthropic - Messages API with tool calling support
- ✅ vLLM - OpenAI-compatible chat completions (local / self-hosted)
Installation
go get github.com/openlit/openlit/sdk/go
Quick Start
1. Initialize OpenLIT
package main
import (
"context"
"log"
"github.com/openlit/openlit/sdk/go"
)
func main() {
// Initialize OpenLIT
err := openlit.Init(openlit.Config{
OtlpEndpoint: "http://127.0.0.1:4318",
Environment: "production",
ApplicationName: "my-go-app",
})
if err != nil {
log.Fatalf("Failed to initialize OpenLIT: %v", err)
}
defer openlit.Shutdown(context.Background())
// Your application code here
}
2. Instrument OpenAI
import (
"github.com/openlit/openlit/sdk/go/instrumentation/openai"
openai_sdk "github.com/sashabaranov/go-openai"
)
// Create and instrument OpenAI client
client := openai_sdk.NewClient("your-api-key")
instrumentedClient := openai.Instrument(client)
// Use as normal - automatically traced!
resp, err := instrumentedClient.CreateChatCompletion(ctx, openai_sdk.ChatCompletionRequest{
Model: openai_sdk.GPT4,
Messages: []openai_sdk.ChatCompletionMessage{
{
Role: openai_sdk.ChatMessageRoleUser,
Content: "Hello!",
},
},
})
3. Instrument Anthropic
import (
"github.com/openlit/openlit/sdk/go/instrumentation/anthropic"
)
// Create and instrument Anthropic client
client := anthropic.NewClient("your-api-key")
instrumentedClient := anthropic.Instrument(client)
// Use as normal - automatically traced!
resp, err := instrumentedClient.CreateMessage(ctx, anthropic.MessageRequest{
Model: "claude-3-5-sonnet-20241022",
Messages: []anthropic.Message{
{
Role: "user",
Content: "Hello!",
},
},
MaxTokens: 1024,
})
4. Instrument vLLM
vLLM exposes an OpenAI-compatible HTTP API. Point the instrumented client at your
vLLM server (default http://127.0.0.1:8000/v1). Spans use gen_ai.system=vllm
so they are distinct from OpenAI.
import (
"github.com/openlit/openlit/sdk/go/instrumentation/vllm"
)
client := vllm.NewClient("http://127.0.0.1:8000/v1") // optional: vllm.WithAPIKey("...")
resp, err := client.CreateChatCompletion(ctx, vllm.ChatCompletionRequest{
Model: "meta-llama/Llama-3-8B-Instruct",
Messages: []vllm.ChatMessage{
{Role: "user", Content: "Hello!"},
},
})
Configuration Options
config := openlit.Config{
// Required
OtlpEndpoint: "http://127.0.0.1:4318", // OTLP endpoint
// Optional
Environment: "production", // Deployment environment
ApplicationName: "my-go-app", // Application name
ServiceVersion: "1.0.0", // Service version
// OTLP Configuration
OtlpHeaders: map[string]string{}, // Custom OTLP headers
// Feature Flags
DisableTracing: false, // Disable tracing
DisableMetrics: false, // Disable metrics
DisableBatch: false, // Disable batch processing
// Timeouts
TraceExporterTimeout: 10 * time.Second, // Trace export timeout
MetricExporterTimeout: 10 * time.Second, // Metric export timeout
MetricExportInterval: 30 * time.Second, // Metric export interval
// Pricing
PricingEndpoint: "", // Custom pricing endpoint
DisablePricingFetch: false, // Disable pricing fetch
PricingInfo: map[string]ModelPricing{}, // Custom pricing
}
Streaming Support
OpenAI, Anthropic, and vLLM integrations support streaming responses:
OpenAI Streaming
stream, err := instrumentedClient.CreateChatCompletionStream(ctx, request)
if err != nil {
log.Fatal(err)
}
defer stream.Close()
for {
response, err := stream.Recv()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
log.Fatal(err)
}
fmt.Printf(response.Choices[0].Delta.Content)
}
Anthropic Streaming
stream, err := instrumentedClient.CreateMessageStream(ctx, request)
if err != nil {
log.Fatal(err)
}
defer stream.Close()
for {
event, err := stream.Recv()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
log.Fatal(err)
}
// Handle event
}
vLLM Streaming
stream, err := client.CreateChatCompletionStream(ctx, vllm.ChatCompletionRequest{
Model: "meta-llama/Llama-3-8B-Instruct",
Messages: []vllm.ChatMessage{{Role: "user", Content: "Hello!"}},
})
if err != nil {
log.Fatal(err)
}
defer stream.Close()
for {
chunk, err := stream.Recv()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
log.Fatal(err)
}
if len(chunk.Choices) > 0 {
fmt.Print(chunk.Choices[0].Delta.Content)
}
}
Collected Telemetry
Traces
- Operation name and type (chat, embedding, etc.)
- Request and response models
- Input/output messages
- Token usage (input, output, total)
- Cost calculations
- Response times
- Error details
Metrics
gen_ai.client.token.usage- Token usage countergen_ai.client.operation.duration- Operation duration histogramgen_ai.server.time_to_first_token- TTFT for streaminggen_ai.server.time_per_output_token- TBT for streaming
Attributes
All traces include standard OpenTelemetry semantic conventions:
gen_ai.operation.namegen_ai.systemgen_ai.request.modelgen_ai.response.modelgen_ai.usage.input_tokensgen_ai.usage.output_tokensserver.addressserver.port
Advanced Usage
Custom Pricing
Provide custom pricing information for models:
config := openlit.Config{
PricingInfo: map[string]openlit.ModelPricing{
"gpt-4-custom": {
InputCostPerToken: 0.00003,
OutputCostPerToken: 0.00006,
},
},
}
Custom Headers
Add custom headers to OTLP exports:
config := openlit.Config{
OtlpHeaders: map[string]string{
"Authorization": "Bearer token",
"X-Custom-Header": "value",
},
}
Integration with OpenLIT Dashboard
The Go SDK works seamlessly with the OpenLIT Dashboard:
- Start OpenLIT stack:
docker compose up -d
- Configure SDK to send data:
openlit.Init(openlit.Config{
OtlpEndpoint: "http://localhost:4318",
})
- View your traces at http://localhost:3000
Examples
See the examples/ directory for complete working examples:
Rule Engine - openlit.EvaluateRule()
Evaluate trace attributes against the OpenLIT Rule Engine to retrieve matching rules and associated entities (contexts, prompts, evaluation configurations). This function does not require openlit.Init() — it is a standalone HTTP call.
Parameters (EvaluateRuleOptions)
| Field | Type | Description |
|---|---|---|
URL |
string |
OpenLIT dashboard URL. Falls back to OPENLIT_URL env, then http://127.0.0.1:3000. |
APIKey |
string |
Bearer token. Falls back to OPENLIT_API_KEY env. |
EntityType |
RuleEntityType |
RuleEntityContext, RuleEntityPrompt, or RuleEntityEvaluation. |
Fields |
map[string]interface{} |
Trace attributes to evaluate against rules. |
IncludeEntityData |
bool |
If true, include full entity data in response. |
EntityInputs |
map[string]interface{} |
Optional inputs for entity resolution (e.g. prompt variables). |
Timeout |
time.Duration |
HTTP timeout. Default: 30s. |
Example
result, err := openlit.EvaluateRule(ctx, openlit.EvaluateRuleOptions{
EntityType: openlit.RuleEntityContext,
Fields: map[string]interface{}{
"gen_ai.system": "openai",
"gen_ai.request.model": "gpt-4",
"service.name": "my-app",
},
IncludeEntityData: true,
})
if err != nil {
log.Fatal(err)
}
fmt.Println("Matching rules:", result.MatchingRuleIDs)
fmt.Println("Entities:", result.Entities)
Requirements
- Go 1.21 or higher
- OpenTelemetry Collector or compatible backend
Contributing
We welcome contributions! See CONTRIBUTING.md for details.
License
Apache License 2.0 - see LICENSE