Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
OpenLIT Controller — Design Principles & Invariants
The controller (openlit-controller/, Go) is a node/cluster agent that:
- Discovers processes talking to LLM endpoints via an eBPF scanner (kprobe on
tcp_connect+/proc/net/tcpconnscan), keyed on destination IP+port. - Instruments discovered workloads two ways: OBI (eBPF payload extraction,
built from upstream + our
patches/) and Python-SDK injection (docker/k8s/systemd/bare). - Reports to the OpenLIT backend via a poll loop; the backend materializes the data into the Agents page.
Full rationale: see openlit-controller/DESIGN.md. The rules below are
invariants — violating one re-introduces a real bug we have already hit.
1. Service naming — the controller is the single naming authority
- A workload's
service.nameis resolved ONCE at discovery (EnrichProcess). Precedence: a user-setOTEL_SERVICE_NAME/service.namein the process env WINS; otherwise container name (docker/k8s) or process name. - All three emission paths must agree on that name: discovery heartbeat, OBI
(
entry.Name = svc.ServiceNamein ALL modes), and SDK injection (OTEL_SERVICE_NAME=svc.ServiceName). If they diverge, traces don't match the Agents summary row and the detail tabs render empty. - The Agents detail tabs match eBPF traces to an agent by
service_nameequality (eBPF spans carry noservice.workload.key), so the names MUST match.
2. Workload identity must be infrastructure-derived, never user-overridable
- The dedup key is
service_name(agent_key = sha1(cluster|env|service_name)) on the controller side, andworkload_keyfor SDK↔controller merge. buildWorkloadKey/stableContainerNamemust derive from infra only (container name → container ID → pod/deployment/namespace). Never fall back tometa.ServiceNamefor the key — it can be overridden byOTEL_SERVICE_NAME, and a key that shifts on env change produces a duplicate agent listing.- Keep the key stable across restarts (so a restart updates the same row). In
Docker/K8s pin identity to container/pod; for the dev compose set a stable
OPENLIT_INSTANCE_ID.
3. Everything from the poll response / config is UNTRUSTED input
- The poll
configand action payloads are attacker-influenceable and are EXECUTED by the controller. Validate at the ingress, before any use:sdk_versionis interpolated intopip install(incl. shell strings) — it MUST match the PEP 440 grammar^[A-Za-z0-9][A-Za-z0-9._+-]*$(validated inEnablePythonSDK, defended again inpypiPackageSpec). Never relax this.- The backend
validateControllerConfigsanitizes the config (types, size caps, control-char strip) before persisting — it is writable by any authenticated user. Keep it in sync withControllerConfig; new config fields must be added there or they are silently dropped. - The poll route coerces/caps every service field before the ClickHouse insert.
4. Never break out of a target sink with unescaped input
- systemd drop-ins:
escapeSystemdValuemust escape\then"then strip CR/LF + control chars (a newline injects a new unit directive). - ClickHouse string literals: use
escapeClickHouseString(the single shared helper insrc/lib/clickhouse-escape.ts) for EVERY interpolated value. Do not re-add per-fileescape()copies. Prefer parameterized inserts (JSONEachRow). - Container selection: match the target container by name exactly; on a specified-but-unmatched name, ERROR — never silently fall back to container[0].
5. OBI subprocess lifecycle
OBIManagerauto-restarts on unexpected exit. An INTENTIONAL stop must set thestoppingflag (under lock) BEFORE signalling SIGTERM —Stop()signals beforecancel(), so the wait goroutine would otherwise seectx.Err()==niland resurrect a just-stopped OBI.Startresets the flag.- OBI config changes (providers, custom hosts, export endpoint) take effect only
via
rebuildOBI; any setter that affects OBI config must call it when running. custom_llm_hostsfeed BOTH the eBPF scanner (discovery) AND OBIGenAI.Custom.Hosts(the host-gated custom extractor). Update both.
6. Concurrency
- Maps shared between the refresh loop and scan loop (e.g.
ConnScannerendpoints) are replaced wholesale under a lock and read via a snapshot — never mutate a shared map in place. - Channel consumers must handle close (
v, ok := <-ch), not spin on zero values. snapshotServiceand any "defensive copy" must deep-copy slices AND maps; a struct copy shares slice backing arrays with the live engine state.
7. OBI patches (patches/ + _obi-providers/)
- We build OBI from a pinned upstream tag (
OBI_BRANCHinDockerfile) and applypatches/{config,span,http_transform}.patch+ vendored_obi-providers/. Bumping the version = re-rebasing the 3 patches; verify theygit applyon a fresh clone and that the obi-builder compiles. - GenAI message attributes MUST follow the strict semantic-conventions structure
(
gen_ai.input.messages/output.messages=[{role, parts:[{type,content}]}], output addsfinish_reason;gen_ai.system_instructions= flat parts array). Operation names are normalized (chat.completion→chat). Span name is"<operation> <model>". - The OBI attribute-select list (
obi.go) must use the EXACT semconv keys (gen_ai.system_instructions, underscore — notgen_ai.system.instructions), or the attribute is silently dropped. - Custom-gateway extraction is detected by the request's destination
host:port matching
custom_llm_hosts— NOT by path or vendor headers. Custom gateways may serve on arbitrary paths (e.g./testchat); do not gate on path.
8. Controller ↔ backend auth (intentional onboarding behavior)
- The poll endpoint requires a valid API key IF any API key exists; if none has been created yet it serves keyless (to speed first-run onboarding). This is deliberate — do NOT "harden" it into always-required; the keyless path closes automatically the moment the first key is created.
9. Robustness
- Backend down / OBI missing / BPF load fail must degrade gracefully (poll backoff; discovery disabled; API still serves) — never crash the controller.
- Destructive workload operations (naked-pod recreate = delete+create) must capture the original spec (logged) and retry rollback; treat the delete→create window as the danger zone.