Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
OpenLIT Kubernetes Example
A complete local Kubernetes setup with all OpenLIT components and sample LLM apps on a 3-node k3d cluster.
Architecture
┌─────────────────────────────────────────────────────────────────────┐
│ k3d Cluster "test" (1 server + 2 agents = 3 nodes) │
│ │
│ ┌──────────────┐ ┌─────────────────────────┐ │
│ │ ClickHouse │◄──│ OpenLIT Dashboard │ │
│ │ (StatefulSet) │ │ + OTLP receiver │ │
│ └──────────────┘ │ (port 3000, 4317, 4318) │ │
│ └────────────┬─────────────┘ │
│ │ │
│ ┌────────────────────────┐ │ │
│ │ OpenLIT Controller │───────┘ │
│ │ (DaemonSet, eBPF) │ discovers │
│ └──────────┬─────────────┘ │
│ │ │
│ ┌──────────┼──────────────────────┐ │
│ │ Sample Apps ▼ │ │
│ │ gemini-app (Pod) │ │
│ │ crewai-agent (Deployment/2) │ │
│ │ bedrock-app (DaemonSet) │ │
│ └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
Components
| Component | Kind | Notes |
|---|---|---|
| ClickHouse | StatefulSet | clickhouse/clickhouse-server:24.4.1 |
| OpenLIT Dashboard | Deployment | Built locally as openlit:local — includes the first-party OTLP receiver |
| OpenLIT Controller | DaemonSet | Built locally as openlit-controller:local — 1 per node |
| gemini-app | naked Pod | Gemini client (tests naked Pod discovery) |
| crewai-agent-app | Deployment (2 replicas) | CrewAI agent using OpenAI (spread across nodes) |
| bedrock-app | DaemonSet | Bedrock client (runs on every node) |
Prerequisites
Quick Start
1. Run the setup script
cd examples/kubernetes
./setup.sh
This will:
- Delete any existing k3d cluster named
test - Create a fresh 3-node k3d cluster (1 server + 2 agents)
- Build the OpenLIT Dashboard and Controller images locally
- Build sample app images (gemini, crewai, bedrock)
- Load all images into the k3d cluster
- Deploy infrastructure (ClickHouse, Dashboard with OTLP receiver)
- Deploy sample apps and wait for them to start making LLM calls
- Deploy the controller last (to test discovery of already-running apps)
2. Access the dashboard
kubectl port-forward svc/openlit -n openlit 3000:3000
Open http://localhost:3000 in your browser.
Default credentials: [email protected] / openlituser
3. Set real API keys (optional)
The sample apps are discovered by the controller even with fake keys (they just produce HTTP errors). To get real LLM traces:
kubectl patch secret llm-api-keys -n openlit -p \
'{"stringData":{"OPENAI_API_KEY":"sk-...","GEMINI_API_KEY":"...","AWS_BEARER_BEDROCK_TOKEN":"..."}}'
# Restart apps to pick up new keys
kubectl delete pod gemini-app -n openlit
kubectl rollout restart deployment/crewai-agent-app -n openlit
kubectl rollout restart daemonset/bedrock-app -n openlit
What to Test
- LLM Observability (eBPF) -- The controller auto-discovers LLM API connections. Check the Agents page to see discovered services and enable/disable instrumentation.
- Agent Observability (SDK injection) -- Enable Agent Observability on any Python service. The controller injects the OpenLIT SDK via init containers and restarts the workload.
- Naked Pod warning --
gemini-appis a naked Pod (no controller). Agent Observability should show "unsupported" since it cannot be restarted safely. - Multi-node -- The controller DaemonSet runs on all 3 nodes. Each instance discovers services on its own node.
- Workload types -- Deployment, DaemonSet, and naked Pod are all represented.
Useful Commands
# Check all pods and their node placement
kubectl get pods -n openlit -o wide
# Follow controller logs (all nodes)
kubectl logs -n openlit -l app=openlit-controller -f
# Follow a specific controller instance
kubectl logs -n openlit -l app=openlit-controller --field-selector spec.nodeName=k3d-test-agent-0 -f
# Check ClickHouse tables
kubectl exec -n openlit statefulset/clickhouse -- \
clickhouse-client --user=default --password=OPENLIT \
--database=openlit --query="SHOW TABLES"
# Port-forward ClickHouse HTTP for debugging
kubectl port-forward svc/clickhouse -n openlit 8123:8123
Teardown
./teardown.sh
# or directly:
k3d cluster delete test