Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
feat: Update LLM instrumentations to align with OTel Semcov (#1002)
Ishan Jain6b14ce22a0f4
diff --git a/contributors/EVENTS_MIGRATION_GUIDE.md b/contributors/EVENTS_MIGRATION_GUIDE.mddeleted file mode 100644index 093b18e9..00000000--- a/contributors/EVENTS_MIGRATION_GUIDE.md+++ /dev/null@@ -1,1064 +0,0 @@-# OpenTelemetry Events Migration Guide--This guide documents the migration pattern from span events to OTel log events for LLM instrumentation, following the [OpenTelemetry Gen-AI Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md).--**Reference Implementation:** OpenAI instrumentation (completed)-**Status:** Production-ready pattern for adoption across all providers-------## Table of Contents--1. [Overview](#overview)-2. [Architecture Decisions](#architecture-decisions)-3. [Implementation Pattern](#implementation-pattern)-4. [Code Examples](#code-examples)-5. [Testing Checklist](#testing-checklist)-6. [Provider-Specific Notes](#provider-specific-notes)-------## Overview--### What Changed--**Before:** Prompt and response content emitted as span events using `span.add_event()`-```python-# OLD - To be removed-span.add_event(- name="gen_ai.content.prompt",- attributes={"gen_ai.prompt": json.dumps(messages)}-)-```--**After:** Content emitted as OTel log events using the events provider-```python-# NEW - Following OTel Gen-AI spec-event = otel_event(- name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: input_messages, # Structured, not JSON string- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: output_messages,- # ... other attributes- },- body=""-)-event_provider.emit(event)-```--### Why This Matters--1. **Compliance:** Aligns with OTel Gen-AI semantic conventions v1.29+-2. **Structure:** Messages as structured objects (not JSON strings)-3. **Standardization:** Consistent event format across all providers-4. **Observability:** Better integration with OTel backends-5. **Future-proof:** Prepares for span events deprecation-------## Architecture Decisions--### 1. Event Name--**Use:** `gen_ai.client.inference.operation.details`-**For:** All operations (chat, completions, embeddings, image, audio)--```python-SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS = "gen_ai.client.inference.operation.details"-```--### 2. Message Format--**Critical:** Use structured objects, NOT JSON strings--```python-# ✅ CORRECT - Structured objects-input_messages = [- {- "role": "user",- "parts": [{"type": "text", "content": "Hello"}]- }-]--# ❌ WRONG - JSON strings-input_messages = json.dumps([...]) # Never do this!-```--### 3. Event Emission Timing--**When:** After receiving complete response (including streaming)--- **Non-streaming:** Emit immediately after response-- **Streaming:** Emit in StopIteration handler after all chunks collected--### 4. Attribute Mapping--**Required Attributes:**-- `gen_ai.operation` (chat, completion, embedding, etc.)-- `gen_ai.request.model`-- `gen_ai.response.model`-- `gen_ai.input.messages` (for chat/completions)-- `gen_ai.output.messages` (for chat/completions)--**Recommended Attributes:**-- `gen_ai.system_instructions` (if system messages present)-- `gen_ai.tool.definitions` (if tools/functions provided)-- `gen_ai.response.id`-- `gen_ai.response.finish_reasons[]`-- `server.address`, `server.port`--**Optional Attributes:**-- `gen_ai.request.temperature`-- `gen_ai.request.max_tokens`-- `gen_ai.request.top_p`-- `gen_ai.request.frequency_penalty`-- `gen_ai.request.presence_penalty`-- `gen_ai.request.stop_sequences[]`-- `gen_ai.request.seed`-- `gen_ai.response.choice_count`-- `gen_ai.response.output_type`-- `gen_ai.usage.input_tokens`-- `gen_ai.usage.output_tokens`--### 5. Backward Compatibility--**Keep existing span attributes** - DO NOT REMOVE:-- `gen_ai.prompt` (legacy)-- `gen_ai.completion` (legacy)-- All metrics-- All existing span attributes--**Only remove:** `span.add_event()` calls-------## Implementation Pattern--### Step-by-Step Migration--#### Phase 1: Add Semantic Conventions--**File:** `src/openlit/semcov/__init__.py`--```python-# Add these constants after existing gen_ai constants-# GenAI Event Names (OTel Semconv v1.29+)-GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS = "gen_ai.client.inference.operation.details"--# GenAI Content Attributes (OTel Semconv v1.29+)-GEN_AI_INPUT_MESSAGES = "gen_ai.input.messages"-GEN_AI_OUTPUT_MESSAGES = "gen_ai.output.messages"-GEN_AI_SYSTEM_INSTRUCTIONS = "gen_ai.system_instructions"-GEN_AI_TOOL_DEFINITIONS = "gen_ai.tool.definitions"-```--#### Phase 2: Create Message Builder Helpers--**File:** `src/openlit/instrumentation/<provider>/utils.py`--**Helper 1: Build Input Messages**-```python-def build_input_messages(messages):- """- Convert provider-specific messages to OTel input message structure.-- Returns:- List of ChatMessage objects with structure:- [- {- "role": "user|assistant|system|tool",- "parts": [- {"type": "text", "content": "..."},- {"type": "uri", "modality": "image", "uri": "..."},- {"type": "tool_call_response", "id": "...", "response": "..."}- ]- }- ]- """- structured_messages = []-- for msg in messages:- # Extract role- role = msg.get("role", "user")-- # Build parts array- parts = []- content = msg.get("content", "")-- if isinstance(content, str):- # Simple text content- parts.append({"type": "text", "content": content})- elif isinstance(content, list):- # Multi-part content (text, images, etc.)- for part in content:- if part.get("type") == "text":- parts.append({"type": "text", "content": part.get("text", "")})- elif part.get("type") == "image_url":- image_url = part.get("image_url", {}).get("url", "")- # Skip data URIs for privacy- if not image_url.startswith("data:"):- parts.append({- "type": "uri",- "modality": "image",- "uri": image_url- })-- # Handle tool calls (for assistant messages)- if "tool_calls" in msg:- for tool_call in msg.get("tool_calls", []):- parts.append({- "type": "tool_call",- "id": tool_call.get("id", ""),- "name": tool_call.get("function", {}).get("name", ""),- "arguments": tool_call.get("function", {}).get("arguments", {})- })-- # Handle tool responses (for tool role)- if role == "tool" and "tool_call_id" in msg:- parts.append({- "type": "tool_call_response",- "id": msg.get("tool_call_id", ""),- "response": content- })-- structured_messages.append({- "role": role,- "parts": parts- })-- return structured_messages-```--**Helper 2: Build Output Messages**-```python-def build_output_messages(response_text, finish_reason, tool_calls=None):- """- Convert provider response to OTel output message structure.-- Returns:- List with single OutputMessage:- [- {- "role": "assistant",- "parts": [- {"type": "text", "content": "..."},- {"type": "tool_call", "id": "...", "name": "...", "arguments": {...}}- ],- "finish_reason": "stop|length|tool_calls|..."- }- ]- """- parts = []-- # Add text content if present- if response_text:- parts.append({"type": "text", "content": response_text})-- # Add tool calls if present- if tool_calls:- for tool_call in tool_calls:- parts.append({- "type": "tool_call",- "id": tool_call.get("id", ""),- "name": tool_call.get("function", {}).get("name", ""),- "arguments": tool_call.get("function", {}).get("arguments", {})- })-- # Map provider finish reasons to OTel standard- finish_reason_map = {- "stop": "stop",- "length": "max_tokens",- "tool_calls": "tool_calls",- "function_call": "tool_calls",- "content_filter": "content_filter",- # Add provider-specific mappings- }-- otel_finish_reason = finish_reason_map.get(finish_reason, finish_reason)-- return [{- "role": "assistant",- "parts": parts,- "finish_reason": otel_finish_reason- }]-```--**Helper 3: Build Tool Definitions**-```python-def build_tool_definitions(tools):- """- Extract tool/function definitions from request.-- Returns:- List of tool definition objects or None:- [- {- "type": "function",- "function": {- "name": "...",- "description": "...",- "parameters": {...}- }- }- ]- """- if not tools:- return None-- # Tools are usually already in correct format- # Just validate and pass through- return tools-```--**Helper 4: Emit Inference Event (Centralized)**-```python-def emit_inference_event(- event_provider,- operation_name,- request_model,- response_model,- input_messages=None,- output_messages=None,- tool_definitions=None,- server_address=None,- server_port=None,- **extra_attrs-):- """- Centralized function to emit gen_ai.client.inference.operation.details event.-- Args:- event_provider: The OTel event provider- operation_name: Operation type (chat, completion, embedding, etc.)- request_model: Model specified in request- response_model: Model returned in response- input_messages: Structured input messages- output_messages: Structured output messages- tool_definitions: Tool/function definitions- server_address: API server address- server_port: API server port- **extra_attrs: Additional attributes (temperature, max_tokens, etc.)- """- try:- if not event_provider:- return-- from openlit.__helpers import otel_event- from openlit.semcov import SemanticConvention-- # Build base attributes- attributes = {- SemanticConvention.GEN_AI_OPERATION: operation_name,- }-- # Add model attributes- if request_model:- attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model- if response_model:- attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model-- # Add server attributes- if server_address:- attributes["server.address"] = server_address- if server_port:- attributes["server.port"] = server_port-- # Add messages (only if not None/empty)- if input_messages:- attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages- if output_messages:- attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages-- # Add tool definitions (recommended)- if tool_definitions:- attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions-- # Map extra attributes to semantic conventions- attr_mapping = {- "temperature": SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- "max_tokens": SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,- "top_p": SemanticConvention.GEN_AI_REQUEST_TOP_P,- "frequency_penalty": SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- "presence_penalty": SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,- "stop_sequences": SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,- "seed": SemanticConvention.GEN_AI_REQUEST_SEED,- "response_id": SemanticConvention.GEN_AI_RESPONSE_ID,- "finish_reasons": SemanticConvention.GEN_AI_RESPONSE_FINISH_REASONS,- "choice_count": SemanticConvention.GEN_AI_RESPONSE_CHOICE_COUNT,- "output_type": SemanticConvention.GEN_AI_RESPONSE_OUTPUT_TYPE,- "input_tokens": SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,- "output_tokens": SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,- }-- for key, value in extra_attrs.items():- if value is not None and key in attr_mapping:- attributes[attr_mapping[key]] = value-- # Create and emit event- event = otel_event(- name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,- attributes=attributes,- body="" # Per spec, all data in attributes- )-- event_provider.emit(event)-- except Exception as e:- import logging- logger = logging.getLogger(__name__)- logger.warning("Failed to emit inference event: %s", e, exc_info=True)-```--#### Phase 3: Update Processing Functions--**File:** `src/openlit/instrumentation/<provider>/utils.py`--**Pattern for common_chat_logic() or equivalent:**--```python-def common_chat_logic(- # ... existing parameters ...- event_provider=None, # NEW PARAMETER-):- """Process chat/completion responses"""-- # ... existing span attribute logic (KEEP THIS) ...-- # Set legacy span attributes- span.set_attribute(SemanticConvention.GEN_AI_PROMPT, formatted_messages)- span.set_attribute(SemanticConvention.GEN_AI_COMPLETION, response_content)-- # REMOVE OLD SPAN EVENTS - Delete these lines:- # span.add_event(name="gen_ai.content.prompt", attributes={...})- # span.add_event(name="gen_ai.content.completion", attributes={...})-- # NEW: Emit log event- if capture_message_content and event_provider:- try:- # Build structured messages- input_msgs = build_input_messages(kwargs.get("messages", []))- output_msgs = build_output_messages(- response_text=llm_response,- finish_reason=finish_reason,- tool_calls=tool_calls- )- tool_defs = build_tool_definitions(kwargs.get("tools"))-- # Emit event with all attributes- emit_inference_event(- event_provider=event_provider,- operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- request_model=request_model,- response_model=response_model,- input_messages=input_msgs,- output_messages=output_msgs,- tool_definitions=tool_defs,- server_address=server_address,- server_port=server_port,- # Add all available request parameters- temperature=kwargs.get("temperature"),- max_tokens=kwargs.get("max_tokens"),- top_p=kwargs.get("top_p"),- frequency_penalty=kwargs.get("frequency_penalty"),- presence_penalty=kwargs.get("presence_penalty"),- stop_sequences=kwargs.get("stop"),- seed=kwargs.get("seed"),- # Add all available response attributes- response_id=response_id,- finish_reasons=[finish_reason],- choice_count=len(choices),- output_type=kwargs.get("response_format", {}).get("type"),- input_tokens=prompt_tokens,- output_tokens=completion_tokens,- )- except Exception as e:- logger.warning("Failed to emit event: %s", e)-```--#### Phase 4: Update Wrapper Functions--**File:** `src/openlit/instrumentation/<provider>/<provider>.py` (sync)--**Pattern for chat_completions wrapper:**--```python-def chat_completions(- # ... existing parameters ...- event_provider=None, # NEW PARAMETER-):- """Wrapper for chat completions"""-- def wrapper(wrapped, instance, args, kwargs):- # ... existing setup ...-- # For streaming responses- class TracedSyncStream:- def __init__(self, wrapped, span, span_name, kwargs,- server_address, server_port, event_provider=None): # NEW- self._wrapped = wrapped- self._span = span- self._event_provider = event_provider # STORE IT- # ... rest of init ...-- def __next__(self):- try:- chunk = next(self._wrapped)- # ... accumulate chunks ...- return chunk- except StopIteration:- # Process complete stream- process_streaming_chat_response(- # ... existing parameters ...- event_provider=self._event_provider, # PASS IT- )- raise-- # Execute wrapped function- response = wrapped(*args, **kwargs)-- # Handle streaming- if isinstance(response, Stream):- return TracedSyncStream(- response, span, span_name, kwargs,- server_address, server_port,- event_provider=event_provider # PASS IT- )-- # Handle non-streaming- process_chat_response(- # ... existing parameters ...- event_provider=event_provider, # PASS IT- )-- return response-- return wrapper-```--**File:** `src/openlit/instrumentation/<provider>/async_<provider>.py` (async)--**Mirror the sync pattern for async:**--```python-def async_chat_completions(- # ... existing parameters ...- event_provider=None, # NEW PARAMETER-):- """Async wrapper for chat completions"""-- async def wrapper(wrapped, instance, args, kwargs):- # ... same pattern as sync ...-- class TracedAsyncStream:- def __init__(self, wrapped, span, span_name, kwargs,- server_address, server_port, event_provider=None): # NEW- self._event_provider = event_provider # STORE IT- # ... rest of init ...-- async def __anext__(self):- try:- chunk = await self._wrapped.__anext__()- # ... accumulate chunks ...- return chunk- except StopAsyncIteration:- # Process complete stream- process_streaming_chat_response(- # ... existing parameters ...- event_provider=self._event_provider, # PASS IT- )- raise-- # ... rest follows sync pattern ...-- return wrapper-```--#### Phase 5: Update Instrumentation Entry Point--**File:** `src/openlit/instrumentation/<provider>/__init__.py`--```python-def _instrument(self, **kwargs):- """Instrument the provider"""-- # Extract event_provider from kwargs- event_provider = kwargs.get("event_provider") # NEW-- # Wrap all operations- wrap_function_wrapper(- "provider_sdk.module",- "ChatCompletions.create",- chat_completions(- # ... existing parameters ...- event_provider=event_provider, # PASS TO WRAPPER- ),- )-- wrap_function_wrapper(- "provider_sdk.module",- "AsyncChatCompletions.create",- async_chat_completions(- # ... existing parameters ...- event_provider=event_provider, # PASS TO WRAPPER- ),- )-- # Repeat for all operations:- # - embeddings (sync + async)- # - image generation (sync + async)- # - audio/speech (sync + async)- # - etc.-```-------## Code Examples--### Example 1: Complete Chat Message Structure--```python-# Input messages with various content types-input_messages = [- {- "role": "system",- "parts": [- {"type": "text", "content": "You are a helpful assistant"}- ]- },- {- "role": "user",- "parts": [- {"type": "text", "content": "What's in this image?"},- {"type": "uri", "modality": "image", "uri": "https://example.com/image.jpg"}- ]- },- {- "role": "assistant",- "parts": [- {"type": "text", "content": "Let me analyze it"},- {- "type": "tool_call",- "id": "call_123",- "name": "analyze_image",- "arguments": {"image_url": "..."}- }- ]- },- {- "role": "tool",- "parts": [- {- "type": "tool_call_response",- "id": "call_123",- "response": "The image contains a cat"- }- ]- }-]--# Output messages-output_messages = [- {- "role": "assistant",- "parts": [- {"type": "text", "content": "Based on the analysis, the image shows a cat."}- ],- "finish_reason": "stop"- }-]-```--### Example 2: Tool Definitions--```python-tool_definitions = [- {- "type": "function",- "function": {- "name": "get_weather",- "description": "Get current weather for a location",- "parameters": {- "type": "object",- "properties": {- "location": {- "type": "string",- "description": "City name"- },- "unit": {- "type": "string",- "enum": ["celsius", "fahrenheit"]- }- },- "required": ["location"]- }- }- }-]-```--### Example 3: Complete Event Emission--```python-emit_inference_event(- event_provider=event_provider,- operation_name="chat",- request_model="gpt-4",- response_model="gpt-4-0613",- input_messages=input_messages,- output_messages=output_messages,- tool_definitions=tool_definitions,- server_address="api.openai.com",- server_port=443,- temperature=0.7,- max_tokens=1000,- top_p=0.9,- response_id="chatcmpl-123",- finish_reasons=["stop"],- choice_count=1,- input_tokens=50,- output_tokens=25-)-```-------## Testing Checklist--### Pre-Migration Testing-- [ ] Identify all `span.add_event()` calls in provider code-- [ ] Document current message formats used by provider-- [ ] Verify existing span attributes are set correctly-- [ ] Run existing test suite - record baseline--### Post-Migration Testing--#### Functional Tests-- [ ] Non-streaming chat/completions with text only-- [ ] Non-streaming with images (if supported)-- [ ] Non-streaming with tool calls-- [ ] Streaming chat/completions-- [ ] Streaming with tool calls-- [ ] Embeddings operations-- [ ] Image generation operations-- [ ] Audio/speech operations--#### Event Validation-- [ ] Event name is `gen_ai.client.inference.operation.details`-- [ ] Input messages in structured format (NOT JSON strings)-- [ ] Output messages in structured format (NOT JSON strings)-- [ ] Messages follow OTel JSON schemas-- [ ] Tool definitions included when tools used-- [ ] All required attributes present-- [ ] All recommended attributes present (when available)-- [ ] Optional attributes included (when available)--#### Backward Compatibility-- [ ] Existing span attributes still set correctly-- [ ] Legacy `gen_ai.prompt` still present-- [ ] Legacy `gen_ai.completion` still present-- [ ] Metrics still recorded-- [ ] No span events present (removed)-- [ ] capture_message_content flag respected-- [ ] event_provider=None works (no events emitted)--#### Error Handling-- [ ] Event emission failure doesn't break instrumentation-- [ ] Malformed messages handled gracefully-- [ ] Missing fields don't cause crashes-- [ ] Non-serializable objects handled--#### Performance-- [ ] No significant latency increase-- [ ] No memory leaks with streaming-- [ ] Event size reasonable (not sending base64 images)-------## Provider-Specific Notes--### Common Provider Variations--#### Message Format Differences--**OpenAI/Azure OpenAI:**-```python-messages = [- {"role": "user", "content": "text"},- {"role": "user", "content": [{"type": "text", "text": "..."}, {"type": "image_url", "image_url": {"url": "..."}}]}-]-```--**Anthropic:**-```python-messages = [- {"role": "user", "content": "text"},- {"role": "user", "content": [{"type": "text", "text": "..."}, {"type": "image", "source": {"type": "url", "url": "..."}}]}-]-```--**Google (Gemini):**-```python-contents = [- {"role": "user", "parts": [{"text": "..."}]},- {"role": "user", "parts": [{"text": "..."}, {"inline_data": {"mime_type": "...", "data": "..."}}]}-]-```--**Cohere:**-```python-message = "text" # Simple string-chat_history = [{"role": "USER", "message": "..."}, {"role": "CHATBOT", "message": "..."}]-```--#### Tool/Function Call Variations--**OpenAI:** `tools` with `function` type-**Anthropic:** `tools` with `function` type (similar)-**Google:** `tools` with `function_declarations`-**Cohere:** `tools` with separate format--**Solution:** Normalize to OTel format in `build_tool_definitions()`--#### Finish Reason Mapping--Each provider has different finish reason names. Create a mapping:--```python-FINISH_REASON_MAP = {- # OpenAI- "stop": "stop",- "length": "max_tokens",- "tool_calls": "tool_calls",- "content_filter": "content_filter",-- # Anthropic- "end_turn": "stop",- "max_tokens": "max_tokens",- "stop_sequence": "stop",-- "STOP": "stop",- "MAX_TOKENS": "max_tokens",- "SAFETY": "content_filter",-- # Cohere- "COMPLETE": "stop",- "MAX_TOKENS": "max_tokens",-- # Add provider-specific mappings-}-```--### Provider Migration Priority--**High Priority (Similar to OpenAI):**-1. Azure OpenAI - Nearly identical API-2. Anthropic - Very similar structure-3. Google Gemini - Similar concepts, different naming--**Medium Priority:**-4. Cohere - Different API structure-5. Mistral - Similar to OpenAI-6. Groq - Similar to OpenAI--**Lower Priority:**-7. HuggingFace - Varies by model-8. Bedrock - Varies by provider--### Common Pitfalls--#### 1. Don't JSON Stringify Messages-```python-# ❌ WRONG-attributes = {- "gen_ai.input.messages": json.dumps(messages)-}--# ✅ CORRECT-attributes = {- "gen_ai.input.messages": messages # Direct structured object-}-```--#### 2. Don't Skip capture_message_content Check-```python-# ❌ WRONG - Always emits-emit_inference_event(event_provider, ...)--# ✅ CORRECT - Check flag first-if capture_message_content and event_provider:- emit_inference_event(event_provider, ...)-```--#### 3. Don't Break on Missing event_provider-```python-# ❌ WRONG - Will crash if None-event_provider.emit(event)--# ✅ CORRECT - Check first-if event_provider:- event_provider.emit(event)-```--#### 4. Don't Include Sensitive Data-```python-# ❌ WRONG - Includes base64 image data-if image_url:- parts.append({"type": "uri", "uri": image_url})--# ✅ CORRECT - Skip data URIs-if image_url and not image_url.startswith("data:"):- parts.append({"type": "uri", "modality": "image", "uri": image_url})-```--#### 5. Don't Remove Existing Span Attributes-```python-# ❌ WRONG - Removes legacy attributes-# span.set_attribute("gen_ai.prompt", messages) # Commented out--# ✅ CORRECT - Keep them for backward compatibility-span.set_attribute("gen_ai.prompt", messages) # Keep this-# Add new events separately-if event_provider:- emit_inference_event(...)-```-------## Evaluation Events Pattern--For evaluation systems (hallucination, bias, toxicity detection):--### Event Name-`gen_ai.evaluation.result`--### Implementation Pattern--```python-# 1. Add semantic conventions-GEN_AI_EVALUATION_RESULT = "gen_ai.evaluation.result"-GEN_AI_EVALUATION_NAME = "gen_ai.evaluation.name"-GEN_AI_EVALUATION_SCORE_VALUE = "gen_ai.evaluation.score.value"-GEN_AI_EVALUATION_SCORE_LABEL = "gen_ai.evaluation.score.label"-GEN_AI_EVALUATION_EXPLANATION = "gen_ai.evaluation.explanation"--# 2. Create helper-def emit_evaluation_event(- event_provider,- evaluation_name,- score_value,- score_label,- explanation,- response_id=None-):- if not event_provider:- return-- attributes = {- SemanticConvention.GEN_AI_EVALUATION_NAME: evaluation_name,- SemanticConvention.GEN_AI_EVALUATION_SCORE_VALUE: float(score_value),- SemanticConvention.GEN_AI_EVALUATION_SCORE_LABEL: score_label,- }-- if explanation:- attributes[SemanticConvention.GEN_AI_EVALUATION_EXPLANATION] = explanation- if response_id:- attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = response_id-- event = otel_event(- name=SemanticConvention.GEN_AI_EVALUATION_RESULT,- attributes=attributes,- body=""- )-- event_provider.emit(event)--# 3. Update evaluator classes-class HallucinationDetector:- def __init__(self, ..., event_provider=None):- self.event_provider = event_provider-- def measure(self, ...):- result = run_evaluation(...)-- # Existing counter metric- if self.collect_metrics:- metric_counter.add(1, attributes)-- # New event emission- if self.event_provider:- emit_evaluation_event(- event_provider=self.event_provider,- evaluation_name="hallucination",- score_value=result.score,- score_label=result.verdict,- explanation=result.explanation- )-- return result-```-------## Reference Links--- [OTel Gen-AI Events Spec](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md)-- [OTel Input Messages Schema](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-input-messages.json)-- [OTel Output Messages Schema](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-output-messages.json)-- [OpenAI Implementation (Reference)](sdk/python/src/openlit/instrumentation/openai/)-------## Migration Checklist Template--Use this checklist when migrating a new provider:--### Planning-- [ ] Read this guide completely-- [ ] Review provider's API documentation-- [ ] Identify all operations to migrate (chat, embeddings, etc.)-- [ ] Map provider message format to OTel structure-- [ ] Map provider parameters to OTel attributes-- [ ] Map provider finish reasons to OTel values--### Implementation-- [ ] Add semantic conventions to `semcov/__init__.py`-- [ ] Create `build_input_messages()` for provider-- [ ] Create `build_output_messages()` for provider-- [ ] Create `build_tool_definitions()` for provider-- [ ] Create `emit_inference_event()` (can reuse pattern)-- [ ] Update all processing functions with event_provider-- [ ] Update all sync wrapper functions-- [ ] Update all async wrapper functions-- [ ] Update instrumentation entry point-- [ ] Remove all `span.add_event()` calls--### Testing-- [ ] Run all tests from Testing Checklist above-- [ ] Verify events in OTel collector/console-- [ ] Validate message structures against JSON schemas-- [ ] Test with real provider API-- [ ] Load test with streaming operations--### Documentation-- [ ] Update provider README with event support-- [ ] Add examples showing event usage-- [ ] Document any provider-specific quirks-- [ ] Update changelog-------## Questions?--If you encounter issues not covered in this guide:--1. Check the OpenAI reference implementation-2. Review the OTel Gen-AI spec-3. Look at `events.py` for event provider setup-4. Check `__helpers.py` for `otel_event()` helper-------**Last Updated:** 2026-02-12-**Reference Implementation:** OpenAI instrumentation (Phases 1-6 complete)-**Status:** Production-ready ✅diff --git a/contributors/OTEL_ATTRIBUTE_MAPPING.md b/contributors/OTEL_ATTRIBUTE_MAPPING.mddeleted file mode 100644index 7d365282..00000000--- a/contributors/OTEL_ATTRIBUTE_MAPPING.md+++ /dev/null@@ -1,722 +0,0 @@-# OTel Gen-AI Attribute Mapping Reference--Quick reference for mapping provider-specific parameters to OpenTelemetry Gen-AI semantic convention attributes.-------## Event Names--| Purpose | OTel Event Name |-|---------|----------------|-| Inference operations (chat, completion, embedding, etc.) | `gen_ai.client.inference.operation.details` |-| Evaluation results (hallucination, bias, toxicity) | `gen_ai.evaluation.result` |-------## Required Attributes (Inference)--| OTel Attribute | Constant | Type | Description | Example |-|----------------|----------|------|-------------|---------|-| `gen_ai.operation.name` | `GEN_AI_OPERATION_NAME` | string | **REQUIRED** - Name of the operation being performed | `"chat"`, `"completion"`, `"embedding"` |-------## Conditionally Required Attributes (Inference)--These attributes MUST be included when the specified condition is met:--| OTel Attribute | Constant | Type | Condition | Description | Example |-|----------------|----------|------|-----------|-------------|---------|-| `error.type` | `ERROR_TYPE` | string | If operation ended in error | Error class/type describing the error | `"timeout"`, `"rate_limit"` |-| `gen_ai.conversation.id` | `GEN_AI_CONVERSATION_ID` | string | When readily available | Identifies the conversation/session | `"session-abc123"` |-| `gen_ai.output.type` | `GEN_AI_OUTPUT_TYPE` | string | When applicable | Requested content type | `"text"`, `"json_object"` |-| `gen_ai.request.choice.count` | `GEN_AI_REQUEST_CHOICE_COUNT` | int | If available and ≠ 1 | Number of choices requested | `3` |-| `gen_ai.request.model` | `GEN_AI_REQUEST_MODEL` | string | If available | Model name being called | `"gpt-4"`, `"claude-3-opus"` |-| `gen_ai.request.seed` | `GEN_AI_REQUEST_SEED` | int | If seed was provided | Seed for deterministic generation | `42` |-| `server.port` | `SERVER_PORT` | int | If server.address is set | Server port number | `443` |-------## Recommended Attributes (Inference)--These attributes SHOULD be included when available:--### Request Parameters--| OTel Attribute | Constant | Type | Description | Example |-|----------------|----------|------|-------------|---------|-| `gen_ai.request.frequency_penalty` | `GEN_AI_REQUEST_FREQUENCY_PENALTY` | double | Frequency penalty setting | `0.5` |-| `gen_ai.request.max_tokens` | `GEN_AI_REQUEST_MAX_TOKENS` | int | Maximum tokens to generate | `1024` |-| `gen_ai.request.presence_penalty` | `GEN_AI_REQUEST_PRESENCE_PENALTY` | double | Presence penalty setting | `0.3` |-| `gen_ai.request.stop_sequences` | `GEN_AI_REQUEST_STOP_SEQUENCES` | string[] | Sequences where model stops | `["END", "\n\n"]` |-| `gen_ai.request.temperature` | `GEN_AI_REQUEST_TEMPERATURE` | double | Temperature setting | `0.7` |-| `gen_ai.request.top_p` | `GEN_AI_REQUEST_TOP_P` | double | Top-p (nucleus) sampling | `0.9` |--### Response Metadata--| OTel Attribute | Constant | Type | Description | Example |-|----------------|----------|------|-------------|---------|-| `gen_ai.response.finish_reasons` | `GEN_AI_RESPONSE_FINISH_REASONS` | string[] | Reasons model stopped | `["stop"]`, `["max_tokens"]` |-| `gen_ai.response.id` | `GEN_AI_RESPONSE_ID` | string | Unique response identifier | `"chatcmpl-123"` |-| `gen_ai.response.model` | `GEN_AI_RESPONSE_MODEL` | string | Model that generated response | `"gpt-4-0613"` |--### Usage Metrics--| OTel Attribute | Constant | Type | Description | Example |-|----------------|----------|------|-------------|---------|-| `gen_ai.usage.cache_creation.input_tokens` | `GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS` | int | Input tokens written to cache | `512` |-| `gen_ai.usage.cache_read.input_tokens` | `GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS` | int | Input tokens served from cache | `256` |-| `gen_ai.usage.input_tokens` | `GEN_AI_USAGE_INPUT_TOKENS` | int | Total tokens in prompt | `128` |-| `gen_ai.usage.output_tokens` | `GEN_AI_USAGE_OUTPUT_TOKENS` | int | Tokens in response | `64` |--### Server Information--| OTel Attribute | Type | Description | Example |-|----------------|------|-------------|---------|-| `server.address` | string | GenAI server hostname/IP | `"api.openai.com"`, `"api.anthropic.com"` |-------## Opt-In Attributes (Content - Chat/Completions)--These attributes contain sensitive content and MUST only be included when explicitly enabled via `capture_message_content=True`:--| OTel Attribute | Constant | Type | Description | Format |-|----------------|----------|------|-------------|--------|-| `gen_ai.input.messages` | `GEN_AI_INPUT_MESSAGES` | array | Chat history provided to model as input | See [Input Message Schema](#input-message-schema) |-| `gen_ai.output.messages` | `GEN_AI_OUTPUT_MESSAGES` | array | Messages returned by model (responses) | See [Output Message Schema](#output-message-schema) |-| `gen_ai.system_instructions` | `GEN_AI_SYSTEM_INSTRUCTIONS` | string or array | System message/instructions separate from chat | `"You are a helpful assistant"` |-| `gen_ai.tool.definitions` | `GEN_AI_TOOL_DEFINITIONS` | array | List of tool/function definitions available | See [Tool Definition Schema](#tool-definition-schema) |--**Important:** These attributes contain user data and MUST only be set when `capture_message_content=True`.-------## Evaluation Event Attributes--### Event: `gen_ai.evaluation.result`--#### Required Attributes--| OTel Attribute | Constant | Type | Description | Example |-|----------------|----------|------|-------------|---------|-| `gen_ai.evaluation.name` | `GEN_AI_EVALUATION_NAME` | string | **REQUIRED** - Evaluation metric name used | `"hallucination"`, `"bias_detection"`, `"toxicity"` |--#### Conditionally Required Attributes--| OTel Attribute | Constant | Type | Condition | Description | Example |-|----------------|----------|------|-----------|-------------|---------|-| `error.type` | `ERROR_TYPE` | string | If operation ended in error | Error type for failed evaluation | `"api_error"` |-| `gen_ai.evaluation.score.label` | `GEN_AI_EVALUATION_SCORE_LABEL` | string | At least one of label/value required | Human-readable evaluation label | `"yes"`, `"no"`, `"pass"`, `"fail"` |-| `gen_ai.evaluation.score.value` | `GEN_AI_EVALUATION_SCORE_VALUE` | double | At least one of label/value required | Numeric evaluation score | `0.85` |--**Note:** Either `score.label` or `score.value` (or both) MUST be present for the evaluation to be meaningful.--#### Recommended Attributes--| OTel Attribute | Constant | Type | Description | Example |-|----------------|----------|------|-------------|---------|-| `gen_ai.evaluation.explanation` | `GEN_AI_EVALUATION_EXPLANATION` | string | Free-form explanation for assigned score | `"The output contains factual inaccuracies"` |-| `gen_ai.response.id` | `GEN_AI_RESPONSE_ID` | string | Links evaluation to specific GenAI response | `"chatcmpl-123"` |-------## Attribute Requirement Levels--Understanding when to include attributes is critical for OTel compliance:--### Required-- **MUST** be included in every event-- Missing required attributes makes the event non-compliant-- Example: `gen_ai.operation.name`--### Conditionally Required-- **MUST** be included when the specified condition is met-- The condition describes when the attribute becomes mandatory-- Examples:- - `error.type` - Required IF the operation ended in error- - `server.port` - Required IF `server.address` is set- - `gen_ai.request.seed` - Required IF a seed was provided in the request--### Recommended-- **SHOULD** be included when the information is available-- Improves observability but not strictly required-- Examples: `gen_ai.response.id`, `gen_ai.usage.input_tokens`--### Opt-In-- **MUST NOT** be included by default (contains sensitive data)-- Only include when user explicitly enables via configuration flag-- Examples: `gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.tool.definitions`-- Check: `if capture_message_content and event_provider:`--### Implementation Guidelines--```python-# Required - always include-attributes = {- "gen_ai.operation.name": "chat", # REQUIRED-}--# Conditionally Required - check condition first-if error_occurred:- attributes["error.type"] = error_type # REQUIRED when condition met--if server_address:- attributes["server.address"] = server_address- attributes["server.port"] = server_port # REQUIRED when server.address set--if seed is not None:- attributes["gen_ai.request.seed"] = seed # REQUIRED when provided--# Recommended - include when available-if response_id:- attributes["gen_ai.response.id"] = response_id # RECOMMENDED--if input_tokens is not None:- attributes["gen_ai.usage.input_tokens"] = input_tokens # RECOMMENDED--# Opt-In - only when explicitly enabled-if capture_message_content and event_provider:- attributes["gen_ai.input.messages"] = input_messages # OPT-IN- attributes["gen_ai.output.messages"] = output_messages # OPT-IN-```-------## Input Message Schema--### Structure-```json-[- {- "role": "user|assistant|system|tool",- "parts": [- {"type": "text", "content": "..."},- {"type": "uri", "modality": "image|audio|video", "uri": "https://..."},- {"type": "tool_call", "id": "...", "name": "...", "arguments": {...}},- {"type": "tool_call_response", "id": "...", "response": "..."}- ]- }-]-```--### Part Types--#### Text Part-```json-{- "type": "text",- "content": "What is the weather?"-}-```--#### URI Part (Image/Audio/Video)-```json-{- "type": "uri",- "modality": "image",- "uri": "https://example.com/image.jpg"-}-```--**Important:** Never include data URIs (`data:image/png;base64,...`). Only HTTP/HTTPS URLs.--#### Tool Call Part (in assistant messages)-```json-{- "type": "tool_call",- "id": "call_123",- "name": "get_weather",- "arguments": {- "location": "San Francisco",- "unit": "celsius"- }-}-```--#### Tool Call Response Part (in tool messages)-```json-{- "type": "tool_call_response",- "id": "call_123",- "response": "The weather is 20°C and sunny"-}-```--### Example: Complete Input Messages--```python-input_messages = [- {- "role": "system",- "parts": [- {"type": "text", "content": "You are a helpful assistant"}- ]- },- {- "role": "user",- "parts": [- {"type": "text", "content": "What's in this image?"},- {"type": "uri", "modality": "image", "uri": "https://example.com/photo.jpg"}- ]- },- {- "role": "assistant",- "parts": [- {"type": "text", "content": "Let me analyze it."},- {- "type": "tool_call",- "id": "call_abc123",- "name": "analyze_image",- "arguments": {"image_url": "https://example.com/photo.jpg"}- }- ]- },- {- "role": "tool",- "parts": [- {- "type": "tool_call_response",- "id": "call_abc123",- "response": "The image shows a cat sitting on a couch"- }- ]- },- {- "role": "user",- "parts": [- {"type": "text", "content": "What breed is it?"}- ]- }-]-```-------## Output Message Schema--### Structure-```json-[- {- "role": "assistant",- "parts": [- {"type": "text", "content": "..."},- {"type": "tool_call", "id": "...", "name": "...", "arguments": {...}}- ],- "finish_reason": "stop|max_tokens|tool_calls|content_filter|..."- }-]-```--**Note:** Output messages array typically contains only ONE message (the assistant's response for choice 0).--### Finish Reasons--Map provider-specific finish reasons to OTel standard values:--| Provider | Provider Value | OTel Value |-|----------|---------------|------------|-| OpenAI | `"stop"` | `"stop"` |-| OpenAI | `"length"` | `"max_tokens"` |-| OpenAI | `"tool_calls"` | `"tool_calls"` |-| OpenAI | `"content_filter"` | `"content_filter"` |-| Anthropic | `"end_turn"` | `"stop"` |-| Anthropic | `"max_tokens"` | `"max_tokens"` |-| Anthropic | `"stop_sequence"` | `"stop"` |-| Google | `"STOP"` | `"stop"` |-| Google | `"MAX_TOKENS"` | `"max_tokens"` |-| Google | `"SAFETY"` | `"content_filter"` |-| Cohere | `"COMPLETE"` | `"stop"` |-| Cohere | `"MAX_TOKENS"` | `"max_tokens"` |--### Example: Output Message--```python-output_messages = [- {- "role": "assistant",- "parts": [- {"type": "text", "content": "Based on the image analysis, it appears to be a British Shorthair cat."}- ],- "finish_reason": "stop"- }-]-```--### Example: Output with Tool Call--```python-output_messages = [- {- "role": "assistant",- "parts": [- {"type": "text", "content": "I'll check the weather for you."},- {- "type": "tool_call",- "id": "call_xyz789",- "name": "get_weather",- "arguments": {- "location": "San Francisco",- "unit": "celsius"- }- }- ],- "finish_reason": "tool_calls"- }-]-```-------## Tool Definition Schema--### Structure-```json-[- {- "type": "function",- "function": {- "name": "function_name",- "description": "What the function does",- "parameters": {- "type": "object",- "properties": {- "param_name": {- "type": "string|number|boolean|object|array",- "description": "Parameter description",- "enum": ["optional", "list", "of", "values"]- }- },- "required": ["param_name"]- }- }- }-]-```--### Example: Weather Function--```python-tool_definitions = [- {- "type": "function",- "function": {- "name": "get_weather",- "description": "Get the current weather for a location",- "parameters": {- "type": "object",- "properties": {- "location": {- "type": "string",- "description": "The city name, e.g., 'San Francisco'"- },- "unit": {- "type": "string",- "enum": ["celsius", "fahrenheit"],- "description": "Temperature unit"- }- },- "required": ["location"]- }- }- }-]-```-------## Provider Parameter Mapping--### OpenAI / Azure OpenAI--Request mapping:-| OpenAI Parameter | OTel Attribute | Required Level | Notes |-|------------------|----------------|----------------|-------|-| `messages` | `gen_ai.input.messages` | Opt-In | Transform to structured format |-| `model` | `gen_ai.request.model` | Conditionally Required | Direct mapping |-| `temperature` | `gen_ai.request.temperature` | Recommended | Direct mapping |-| `max_tokens` / `max_completion_tokens` | `gen_ai.request.max_tokens` | Recommended | Direct mapping |-| `top_p` | `gen_ai.request.top_p` | Recommended | Direct mapping |-| `frequency_penalty` | `gen_ai.request.frequency_penalty` | Recommended | Direct mapping |-| `presence_penalty` | `gen_ai.request.presence_penalty` | Recommended | Direct mapping |-| `stop` | `gen_ai.request.stop_sequences` | Recommended | Direct mapping |-| `seed` | `gen_ai.request.seed` | Conditionally Required | Include if provided |-| `n` | `gen_ai.request.choice.count` | Conditionally Required | Include if ≠ 1 |-| `tools` / `functions` | `gen_ai.tool.definitions` | Opt-In | Direct mapping |-| `response_format.type` | `gen_ai.output.type` | Conditionally Required | When applicable |--Response mapping:-| OpenAI Field | OTel Attribute | Required Level | Notes |-|--------------|----------------|----------------|-------|-| `id` | `gen_ai.response.id` | Recommended | Direct mapping |-| `model` | `gen_ai.response.model` | Recommended | Direct mapping |-| `choices[0].message.content` | Part of `gen_ai.output.messages` | Opt-In | Text content |-| `choices[0].message.tool_calls` | Part of `gen_ai.output.messages` | Opt-In | Tool calls |-| `choices[0].finish_reason` | Part of `gen_ai.output.messages[0].finish_reason` | Recommended | Map: length→max_tokens |-| `usage.prompt_tokens` | `gen_ai.usage.input_tokens` | Recommended | Direct mapping |-| `usage.completion_tokens` | `gen_ai.usage.output_tokens` | Recommended | Direct mapping |-| `usage.prompt_tokens_details.cached_tokens` | `gen_ai.usage.cache_read.input_tokens` | Recommended | Cached input tokens |-| `system_fingerprint` | Custom attribute | Optional | OpenAI-specific backend configuration identifier |--### Anthropic--Request mapping:-| Anthropic Parameter | OTel Attribute | Required Level | Notes |-|---------------------|----------------|----------------|-------|-| `messages` | `gen_ai.input.messages` | Opt-In | Transform to structured format |-| `system` | `gen_ai.system_instructions` | Opt-In | Single string (not in messages array) |-| `model` | `gen_ai.request.model` | Conditionally Required | Direct mapping |-| `temperature` | `gen_ai.request.temperature` | Recommended | Direct mapping |-| `max_tokens` | `gen_ai.request.max_tokens` | Recommended | Direct mapping |-| `top_p` | `gen_ai.request.top_p` | Recommended | Direct mapping |-| `top_k` | `gen_ai.request.top_k` | Recommended | Direct mapping (Anthropic-specific) |-| `stop_sequences` | `gen_ai.request.stop_sequences` | Recommended | Direct mapping |-| `tools` | `gen_ai.tool.definitions` | Opt-In | Transform if needed |--Response mapping:-| Anthropic Field | OTel Attribute | Required Level |-|-----------------|----------------|----------------|-| `id` | `gen_ai.response.id` | Recommended |-| `model` | `gen_ai.response.model` | Recommended |-| `content[0].text` | Part of `gen_ai.output.messages` | Opt-In |-| `content[*].type="tool_use"` | Part of `gen_ai.output.messages` | Opt-In |-| `stop_reason` | Map to `gen_ai.output.messages[0].finish_reason` | Recommended |-| `usage.input_tokens` | `gen_ai.usage.input_tokens` | Recommended |-| `usage.output_tokens` | `gen_ai.usage.output_tokens` | Recommended |-| `usage.cache_creation_input_tokens` | `gen_ai.usage.cache_creation.input_tokens` | Recommended |-| `usage.cache_read_input_tokens` | `gen_ai.usage.cache_read.input_tokens` | Recommended |--### Google (Gemini)--Request mapping:-| Google Parameter | OTel Attribute | Required Level | Notes |-|------------------|----------------|----------------|-------|-| `contents` | `gen_ai.input.messages` | Opt-In | Already uses "parts" structure, map role "model"→"assistant" |-| `systemInstruction` | `gen_ai.system_instructions` | Opt-In | Direct mapping |-| `model` | `gen_ai.request.model` | Conditionally Required | Extract from full name (e.g., "models/gemini-pro" → "gemini-pro") |-| `generationConfig.temperature` | `gen_ai.request.temperature` | Recommended | Direct mapping |-| `generationConfig.maxOutputTokens` | `gen_ai.request.max_tokens` | Recommended | Direct mapping |-| `generationConfig.topP` | `gen_ai.request.top_p` | Recommended | Direct mapping |-| `generationConfig.topK` | `gen_ai.request.top_k` | Recommended | Direct mapping |-| `generationConfig.stopSequences` | `gen_ai.request.stop_sequences` | Recommended | Direct mapping |-| `generationConfig.seed` | `gen_ai.request.seed` | Conditionally Required | Include if provided |-| `tools` | `gen_ai.tool.definitions` | Opt-In | Transform `function_declarations` |--Response mapping:-| Google Field | OTel Attribute | Required Level | Notes |-|--------------|----------------|----------------|-------|-| N/A | `gen_ai.response.id` | Recommended | Google doesn't provide response IDs - set to None |-| Extracted from request | `gen_ai.response.model` | Recommended | Use request model |-| `candidates[0].content.parts[0].text` | Part of `gen_ai.output.messages` | Opt-In | Text content |-| `candidates[0].content.parts[*].functionCall` | Part of `gen_ai.output.messages` | Opt-In | Tool calls (no IDs provided) |-| `candidates[0].finishReason` | Map to `gen_ai.output.messages[0].finish_reason` | Recommended | Map: STOP→stop, MAX_TOKENS→max_tokens, SAFETY→content_filter |-| `usageMetadata.promptTokenCount` | `gen_ai.usage.input_tokens` | Recommended | Direct mapping |-| `usageMetadata.candidatesTokenCount` | `gen_ai.usage.output_tokens` | Recommended | Direct mapping |-| `usageMetadata.cachedContentTokenCount` | `gen_ai.usage.cache_read.input_tokens` | Recommended | Cached input tokens |-| `usageMetadata.thoughtsTokenCount` | Custom attribute | Recommended | Google-specific reasoning tokens (not in OTel spec yet) |--### Cohere--Request mapping:-| Cohere Parameter | OTel Attribute | Required Level | Notes |-|------------------|----------------|----------------|-------|-| `message` | `gen_ai.input.messages` | Opt-In | Transform to array with user role |-| `chat_history` | `gen_ai.input.messages` | Opt-In | Combine with message |-| `preamble` | `gen_ai.system_instructions` | Opt-In | Direct mapping |-| `model` | `gen_ai.request.model` | Conditionally Required | Direct mapping |-| `temperature` | `gen_ai.request.temperature` | Recommended | Direct mapping |-| `max_tokens` | `gen_ai.request.max_tokens` | Recommended | Direct mapping |-| `p` | `gen_ai.request.top_p` | Recommended | Direct mapping (Cohere uses 'p' instead of 'top_p') |-| `k` | `gen_ai.request.top_k` | Recommended | Direct mapping (Cohere uses 'k' instead of 'top_k') |-| `stop_sequences` | `gen_ai.request.stop_sequences` | Recommended | Direct mapping |-| `seed` | `gen_ai.request.seed` | Conditionally Required | Include if provided |-| `tools` | `gen_ai.tool.definitions` | Opt-In | Transform format |--Response mapping:-| Cohere Field | OTel Attribute | Required Level | Notes |-|--------------|----------------|----------------|-------|-| `generation_id` | `gen_ai.response.id` | Recommended | Direct mapping |-| From request | `gen_ai.response.model` | Recommended | Echo request model |-| `text` | Part of `gen_ai.output.messages` | Opt-In | Text content |-| `tool_calls` | Part of `gen_ai.output.messages` | Opt-In | Tool calls |-| `finish_reason` | Map to `gen_ai.output.messages[0].finish_reason` | Recommended | Map: COMPLETE→stop, MAX_TOKENS→max_tokens |-| `meta.tokens.input_tokens` | `gen_ai.usage.input_tokens` | Recommended | Direct mapping |-| `meta.tokens.output_tokens` | `gen_ai.usage.output_tokens` | Recommended | Direct mapping |-------## Missing Attributes to Add--Based on the OTel Gen-AI semantic conventions, these attributes are currently missing from our implementation and should be added:--### High Priority (Conditionally Required)--| Attribute | Type | Condition | Current Status | Implementation Notes |-|-----------|------|-----------|----------------|---------------------|-| `gen_ai.conversation.id` | string | When readily available | ❌ Not implemented | Add when session/conversation tracking is available |-| `gen_ai.output.type` | string | When applicable | ❌ Not implemented | Extract from `response_format.type` (OpenAI), add to emit_inference_event() |-| `gen_ai.request.choice.count` | int | If available and ≠ 1 | ❌ Not implemented | Extract from `n` parameter (OpenAI), add to emit_inference_event() |-| `error.type` | string | If operation failed | ❌ Not implemented | Capture error class in exception handlers, add to event attributes |--### Medium Priority (Recommended)--| Attribute | Type | Current Status | Implementation Notes |-|-----------|------|----------------|---------------------|-| `gen_ai.request.frequency_penalty` | double | ✅ Partially | Only in OpenAI, add to Anthropic/Google event emission |-| `gen_ai.request.presence_penalty` | double | ✅ Partially | Only in OpenAI, add to event emission |-| `gen_ai.usage.cache_creation.input_tokens` | int | ❌ Not implemented | Anthropic provides this, add to emit_inference_event() |-| `gen_ai.usage.cache_read.input_tokens` | int | ❌ Not implemented | Anthropic & OpenAI provide this, add to emit_inference_event() |--### Implementation Checklist--**For OpenAI:**-- [ ] Add `gen_ai.output.type` from `response_format.type`-- [ ] Add `gen_ai.request.choice.count` from `n` parameter-- [ ] Add `gen_ai.usage.cache_read.input_tokens` from `usage.prompt_tokens_details.cached_tokens`-- [ ] Add `gen_ai.request.frequency_penalty` to event emission-- [ ] Add `gen_ai.request.presence_penalty` to event emission-- [ ] Add error handling with `error.type` attribute--**For Anthropic:**-- [ ] Add `gen_ai.usage.cache_creation.input_tokens` from `usage.cache_creation_input_tokens`-- [ ] Add `gen_ai.usage.cache_read.input_tokens` from `usage.cache_read_input_tokens`-- [ ] Add error handling with `error.type` attribute--**For Google AI Studio:**-- [ ] Add `gen_ai.usage.cache_read.input_tokens` from `usageMetadata.cachedContentTokenCount`-- [ ] Add `gen_ai.request.seed` when provided-- [ ] Add error handling with `error.type` attribute--**All Providers:**-- [ ] Add `gen_ai.conversation.id` extraction when session tracking is available-- [ ] Implement error.type capture in exception handlers-- [ ] Update emit_inference_event() signature to accept new attributes-- [ ] Update SemanticConvention constants for new attributes-------## Common Patterns--### Pattern 1: Extract Request Attributes--```python-def extract_request_attributes(kwargs):- """Extract all available request parameters"""- return {- "temperature": kwargs.get("temperature"),- "max_tokens": kwargs.get("max_tokens"),- "top_p": kwargs.get("top_p"),- "frequency_penalty": kwargs.get("frequency_penalty"),- "presence_penalty": kwargs.get("presence_penalty"),- "stop_sequences": kwargs.get("stop"),- "seed": kwargs.get("seed"),- }-```--### Pattern 2: Extract Response Attributes--```python-def extract_response_attributes(response):- """Extract all available response metadata"""- return {- "response_id": response.id,- "finish_reasons": [choice.finish_reason for choice in response.choices],- "choice_count": len(response.choices),- "output_type": response.object, # or from request- "input_tokens": response.usage.prompt_tokens,- "output_tokens": response.usage.completion_tokens,- }-```--### Pattern 3: Combine Attributes for Event--```python-# In processing function-request_attrs = extract_request_attributes(kwargs)-response_attrs = extract_response_attributes(response)--emit_inference_event(- event_provider=event_provider,- operation_name=operation_type,- request_model=request_model,- response_model=response_model,- input_messages=input_messages,- output_messages=output_messages,- tool_definitions=tool_definitions,- server_address=server_address,- server_port=443,- **request_attrs,- **response_attrs-)-```-------## Validation Checklist--When implementing event emission, verify:--### Event Structure-- [ ] Event name is exactly `gen_ai.client.inference.operation.details` (inference) or `gen_ai.evaluation.result` (evaluation)-- [ ] Event body is empty string (all data in attributes per OTel spec)-- [ ] Using OTel EventLoggerProvider (not span events)--### Required Attributes-- [ ] `gen_ai.operation.name` always present (inference events)-- [ ] `gen_ai.evaluation.name` always present (evaluation events)--### Conditionally Required Attributes-- [ ] `error.type` included if operation failed-- [ ] `server.port` included if `server.address` is set-- [ ] `gen_ai.request.seed` included if seed was provided-- [ ] `gen_ai.evaluation.score.label` or `gen_ai.evaluation.score.value` present (at least one)--### Recommended Attributes-- [ ] Request parameters included when available (`temperature`, `max_tokens`, `top_p`, etc.)-- [ ] Response metadata included when available (`response_id`, `finish_reasons`, `response_model`)-- [ ] Usage metrics included when available (`input_tokens`, `output_tokens`, cache tokens)-- [ ] Server information included when known (`server.address`, `server.port`)--### Opt-In Attributes (Content)-- [ ] Content attributes ONLY included when `capture_message_content=True`-- [ ] Input messages are structured objects (not JSON strings)-- [ ] Output messages are structured objects (not JSON strings)-- [ ] No data URIs in image references (only HTTP/HTTPS URLs)-- [ ] Tool definitions included when tools are used-- [ ] System instructions included when present--### Message Format-- [ ] Input messages follow [OTel input schema](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-input-messages.json)-- [ ] Output messages follow [OTel output schema](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-output-messages.json)-- [ ] Finish reasons mapped to OTel standard values (not provider-specific)-- [ ] Message parts use correct types: `text`, `uri`, `tool_call`, `tool_call_response`--### Error Handling-- [ ] Event emission wrapped in try/except (never raises)-- [ ] Silent failures with warning logs-- [ ] Instrumentation continues even if event emission fails-------## References--- [OTel Gen-AI Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md)-- [OTel Gen-AI Attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/README.md)-- [Input Messages JSON Schema](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-input-messages.json)-- [Output Messages JSON Schema](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-output-messages.json)-------## Summary of Requirement Levels--| Level | Symbol | Meaning | Example |-|-------|--------|---------|---------|-| **Required** | 🔴 | MUST be present in every event | `gen_ai.operation.name` |-| **Conditionally Required** | 🟡 | MUST be present when condition met | `error.type` (if error occurred) |-| **Recommended** | 🟢 | SHOULD be present when available | `gen_ai.response.id`, `gen_ai.usage.input_tokens` |-| **Opt-In** | 🔵 | Only when explicitly enabled (sensitive data) | `gen_ai.input.messages`, `gen_ai.output.messages` |-------**Last Updated:** 2026-02-12 (Expanded with complete OTel attribute requirements)diff --git a/contributors/PROVIDER_MIGRATION_CHECKLIST.md b/contributors/PROVIDER_MIGRATION_CHECKLIST.mddeleted file mode 100644index f8715710..00000000--- a/contributors/PROVIDER_MIGRATION_CHECKLIST.md+++ /dev/null@@ -1,421 +0,0 @@-# Provider Events Migration Checklist--**Provider Name:** `_________________`-**Date Started:** `_________________`-**Assignee:** `_________________`-**Status:** `[ ] Not Started` `[ ] In Progress` `[ ] Testing` `[ ] Complete`-------## Phase 1: Planning & Analysis--### Provider Research-- [ ] Reviewed provider API documentation-- [ ] Identified all supported operations:- - [ ] Chat/Completions- - [ ] Embeddings- - [ ] Image Generation- - [ ] Audio/Speech- - [ ] Other: `________________`-- [ ] Documented message format differences-- [ ] Created parameter mapping table-- [ ] Created finish reason mapping table--### Current State Analysis-- [ ] Found all existing `span.add_event()` calls- - Location 1: `_______________________` Line: `____`- - Location 2: `_______________________` Line: `____`- - Location 3: `_______________________` Line: `____`-- [ ] Documented current span attributes-- [ ] Ran existing test suite (baseline)- - Tests passed: `____` Failed: `____`-------## Phase 2: Semantic Conventions--### File: `src/openlit/semcov/__init__.py`--- [ ] Added `GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS`-- [ ] Added `GEN_AI_INPUT_MESSAGES`-- [ ] Added `GEN_AI_OUTPUT_MESSAGES`-- [ ] Added `GEN_AI_SYSTEM_INSTRUCTIONS` (if applicable)-- [ ] Added `GEN_AI_TOOL_DEFINITIONS` (if applicable)-- [ ] Added provider-specific constants (if needed)- - [ ] `_______________________`-------## Phase 3: Helper Functions--### File: `src/openlit/instrumentation/<provider>/utils.py`--#### build_input_messages()-- [ ] Created function signature-- [ ] Handles text messages-- [ ] Handles multi-part messages-- [ ] Handles images (URI only, no data URIs)-- [ ] Handles tool call responses-- [ ] Handles provider-specific message types:- - [ ] `_______________________`-- [ ] Returns structured objects (not JSON strings)-- [ ] Added error handling-- [ ] Added unit tests--#### build_output_messages()-- [ ] Created function signature-- [ ] Handles text responses-- [ ] Handles tool calls-- [ ] Maps finish reasons correctly-- [ ] Returns structured objects (not JSON strings)-- [ ] Added error handling-- [ ] Added unit tests--#### build_tool_definitions()-- [ ] Created function signature-- [ ] Extracts tool definitions from request-- [ ] Normalizes to OTel format-- [ ] Handles provider-specific format:- - [ ] `_______________________`-- [ ] Returns None if no tools-- [ ] Added error handling-- [ ] Added unit tests--#### emit_inference_event()-- [ ] Created centralized emission function-- [ ] Maps all required attributes-- [ ] Maps all recommended attributes-- [ ] Maps optional attributes-- [ ] Uses `otel_event()` helper-- [ ] Includes comprehensive error handling-- [ ] Never raises exceptions-- [ ] Logs warnings on failure-- [ ] Added unit tests-------## Phase 4: Processing Functions--### Update common logic functions--#### Chat/Completions Processing-- [ ] Added `event_provider` parameter-- [ ] **KEPT** existing span attributes-- [ ] **REMOVED** `span.add_event()` calls-- [ ] Added event emission with `capture_message_content` check-- [ ] Builds input messages-- [ ] Builds output messages-- [ ] Builds tool definitions-- [ ] Passes all attributes to `emit_inference_event()`-- [ ] Added error handling--#### Embeddings Processing (if applicable)-- [ ] Added `event_provider` parameter-- [ ] Added event emission logic-- [ ] Handles single text input-- [ ] Handles array of texts-- [ ] Added error handling--#### Image Generation Processing (if applicable)-- [ ] Added `event_provider` parameter-- [ ] Added event emission logic-- [ ] Handles text prompts-- [ ] Handles revised prompts (output)-- [ ] Added error handling--#### Audio/Speech Processing (if applicable)-- [ ] Added `event_provider` parameter-- [ ] Added event emission logic-- [ ] Handles text input-- [ ] Handles audio output references-- [ ] Added error handling-------## Phase 5: Wrapper Functions (Sync)--### File: `src/openlit/instrumentation/<provider>/<provider>.py`--#### chat_completions()-- [ ] Added `event_provider` parameter to wrapper factory-- [ ] Updated `TracedSyncStream.__init__()` to accept event_provider-- [ ] Stored event_provider in TracedSyncStream instance-- [ ] Passed event_provider in StopIteration handler-- [ ] Passed event_provider in non-streaming branch-- [ ] Tested streaming responses-- [ ] Tested non-streaming responses--#### embeddings()-- [ ] Added `event_provider` parameter-- [ ] Passed to processing function-- [ ] Tested with single text-- [ ] Tested with text array--#### image_generation()-- [ ] Added `event_provider` parameter-- [ ] Passed to processing function-- [ ] Tested single image-- [ ] Tested multiple images--#### audio_create()-- [ ] Added `event_provider` parameter-- [ ] Passed to processing function-- [ ] Tested text-to-speech--#### Other operations:-- [ ] `_________________`: Added event_provider-- [ ] `_________________`: Added event_provider-------## Phase 6: Wrapper Functions (Async)--### File: `src/openlit/instrumentation/<provider>/async_<provider>.py`--#### async_chat_completions()-- [ ] Added `event_provider` parameter to wrapper factory-- [ ] Updated `TracedAsyncStream.__init__()` to accept event_provider-- [ ] Stored event_provider in TracedAsyncStream instance-- [ ] Passed event_provider in StopAsyncIteration handler-- [ ] Passed event_provider in non-streaming branch-- [ ] Tested async streaming responses-- [ ] Tested async non-streaming responses--#### async_embeddings()-- [ ] Added `event_provider` parameter-- [ ] Passed to processing function-- [ ] Tested async embeddings--#### async_image_generation()-- [ ] Added `event_provider` parameter-- [ ] Passed to processing function-- [ ] Tested async image generation--#### async_audio_create()-- [ ] Added `event_provider` parameter-- [ ] Passed to processing function-- [ ] Tested async audio--#### Other async operations:-- [ ] `_________________`: Added event_provider-- [ ] `_________________`: Added event_provider-------## Phase 7: Instrumentation Entry Point--### File: `src/openlit/instrumentation/<provider>/__init__.py`--- [ ] Extracted `event_provider = kwargs.get("event_provider")`-- [ ] Passed to all sync wrapper calls:- - [ ] chat_completions- - [ ] embeddings- - [ ] image_generation- - [ ] audio_create- - [ ] Other: `_________________`-- [ ] Passed to all async wrapper calls:- - [ ] async_chat_completions- - [ ] async_embeddings- - [ ] async_image_generation- - [ ] async_audio_create- - [ ] Other: `_________________`-------## Phase 8: Cleanup--- [ ] Verified all `span.add_event()` calls removed- - Remaining: `____` (should be 0)-- [ ] Verified all span attributes still present-- [ ] Removed any unused imports-- [ ] Ran linter/formatter-- [ ] Updated code comments-------## Phase 9: Testing--### Unit Tests-- [ ] Added tests for `build_input_messages()`- - [ ] Text messages- - [ ] Multi-part messages- - [ ] Images (URIs)- - [ ] Tool responses- - [ ] Edge cases-- [ ] Added tests for `build_output_messages()`- - [ ] Text responses- - [ ] Tool calls- - [ ] Finish reasons- - [ ] Edge cases-- [ ] Added tests for `build_tool_definitions()`- - [ ] Function definitions- - [ ] None/empty tools-- [ ] Added tests for `emit_inference_event()`- - [ ] All attributes present- - [ ] Error handling--### Integration Tests--#### Chat/Completions-- [ ] Non-streaming text only-- [ ] Non-streaming with images (if supported)-- [ ] Non-streaming with tools-- [ ] Non-streaming with tool responses-- [ ] Streaming text-- [ ] Streaming with tools-- [ ] Multi-message conversation-- [ ] System messages--#### Embeddings (if applicable)-- [ ] Single text-- [ ] Multiple texts-- [ ] Token array input--#### Image Generation (if applicable)-- [ ] Single image-- [ ] Multiple images-- [ ] With revised prompts--#### Audio (if applicable)-- [ ] Text-to-speech-- [ ] Different voices/models--### Event Validation-- [ ] Event name is `gen_ai.client.inference.operation.details`-- [ ] Input messages are structured (not JSON strings)-- [ ] Output messages are structured (not JSON strings)-- [ ] Messages follow OTel JSON schema format-- [ ] Tool definitions included when present-- [ ] All required attributes present:- - [ ] `gen_ai.operation`- - [ ] `gen_ai.request.model`- - [ ] `gen_ai.response.model`-- [ ] All recommended attributes present (when available):- - [ ] `gen_ai.system_instructions`- - [ ] `gen_ai.tool.definitions`- - [ ] `gen_ai.response.id`- - [ ] `gen_ai.response.finish_reasons`- - [ ] `server.address`- - [ ] `server.port`-- [ ] Optional attributes included (when available):- - [ ] Request parameters (temperature, max_tokens, etc.)- - [ ] Response metadata (choice_count, output_type)- - [ ] Usage tokens (input_tokens, output_tokens)--### Backward Compatibility-- [ ] Existing span attributes still set:- - [ ] `gen_ai.prompt`- - [ ] `gen_ai.completion`- - [ ] All provider-specific attributes-- [ ] Metrics still recorded correctly-- [ ] No span events present (all removed)-- [ ] `capture_message_content=False` skips events-- [ ] `event_provider=None` skips events-- [ ] Existing functionality unchanged--### Error Handling-- [ ] Event emission failure doesn't break instrumentation-- [ ] Malformed messages handled gracefully-- [ ] Missing fields don't cause crashes-- [ ] Non-serializable objects handled-- [ ] Exceptions logged but not raised--### Performance Testing-- [ ] No significant latency increase- - Baseline: `____ms` New: `____ms`-- [ ] No memory leaks with streaming-- [ ] Event size reasonable (<100KB per event)-- [ ] No base64 images in events--### Manual Verification-- [ ] Events visible in console exporter-- [ ] Events visible in OTLP collector-- [ ] Message structure validated against JSON schema-- [ ] Tested with real provider API-- [ ] Tested with rate limits/errors-------## Phase 10: Documentation--- [ ] Updated provider README with event support-- [ ] Added example code showing event usage-- [ ] Documented provider-specific message format differences-- [ ] Documented finish reason mappings-- [ ] Added troubleshooting section-- [ ] Updated CHANGELOG.md-- [ ] Added migration notes (if breaking changes)-------## Phase 11: Code Review--- [ ] Self-review completed-- [ ] Peer review requested-- [ ] All review comments addressed-- [ ] CI/CD pipeline passing-- [ ] Test coverage maintained/improved- - Previous: `____%` New: `____%`-------## Phase 12: Deployment--- [ ] Merged to main branch-- [ ] Version bumped (if applicable)-- [ ] Release notes prepared-- [ ] Released to production-- [ ] Monitoring dashboards updated-- [ ] Announced to team/users-------## Notes & Issues--### Provider-Specific Quirks-```-Document any unusual behavior, workarounds, or special handling required:--1.--2.--3.-```--### Open Questions-```-List any unresolved questions or items needing clarification:--1.--2.--3.-```--### Known Issues-```-Document any known issues or limitations:--1.--2.--3.-```-------## Sign-Off--- [ ] Developer: `_________________` Date: `_________`-- [ ] Reviewer: `_________________` Date: `_________`-- [ ] QA: `_________________` Date: `_________`-------**Reference Documentation:**-- [Events Migration Guide](./EVENTS_MIGRATION_GUIDE.md)-- [OTel Gen-AI Events Spec](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md)-- [OpenAI Reference Implementation](./src/openlit/instrumentation/openai/)diff --git a/contributors/adding-new-vectordb-instrumentation.md b/contributors/adding-new-vectordb-instrumentation.mddeleted file mode 100644index f5fb8896..00000000--- a/contributors/adding-new-vectordb-instrumentation.md+++ /dev/null@@ -1,608 +0,0 @@-# Vector Database Instrumentation Guide--This guide provides step-by-step instructions for adding a new vector database instrumentation to OpenLIT, following the established patterns and conventions learned from Pinecone and ChromaDB implementations.--## Table of Contents-1. [File Structure](#file-structure)-2. [Code Structure](#code-structure)-3. [Naming Conventions](#naming-conventions)-4. [Code Style Guidelines](#code-style-guidelines)-5. [Implementation Steps](#implementation-steps)-6. [Testing](#testing)-7. [Integration](#integration)--## File Structure--Every vector database instrumentation follows a consistent **3-4 file structure**:--```-sdk/python/src/openlit/instrumentation/{database_name}/-├── __init__.py # Instrumentor class with dependency requirements-├── {database_name}.py # Sync wrapper functions-├── async_{database_name}.py # Async wrapper functions (only if database supports async)-└── utils.py # Telemetry processing logic-```--### Example Structure (using "pinecone" as example):-```-sdk/python/src/openlit/instrumentation/pinecone/-├── __init__.py-├── pinecone.py-├── async_pinecone.py # Only if async operations exist-└── utils.py-```--**Note**: If the database doesn't support async operations (like ChromaDB), omit the `async_{database_name}.py` file.--## Code Structure--### 1. DB_OPERATION_MAP Pattern--All vector database instrumentations use a centralized operation mapping in `utils.py`:--```python-# Operation mapping for simple span naming-DB_OPERATION_MAP = {- "{database}.create_collection": SemanticConvention.DB_OPERATION_CREATE_COLLECTION,- "{database}.add": SemanticConvention.DB_OPERATION_INSERT,- "{database}.get": SemanticConvention.DB_OPERATION_GET,- "{database}.query": SemanticConvention.DB_OPERATION_GET, # Both get() and query() are GET operations- "{database}.update": SemanticConvention.DB_OPERATION_UPDATE,- "{database}.upsert": SemanticConvention.DB_OPERATION_UPSERT,- "{database}.delete": SemanticConvention.DB_OPERATION_DELETE,-}-```--**Key Learning**: Multiple endpoints can map to the same semantic operation. Use the `endpoint` parameter to differentiate implementation details.--### 2. General Wrap Pattern--Use the **general_wrap** pattern for efficiency:--```python-def general_wrap(gen_ai_endpoint, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics):- """- Create a wrapper for {database} operations using the general wrap pattern.- """- def wrapper(wrapped, instance, args, kwargs):- # CRITICAL: Suppression check- if context_api.get_value(context_api._SUPPRESS_INSTRUMENTATION_KEY):- return wrapped(*args, **kwargs)-- # Get server address and port using the standard helper- server_address, server_port = set_server_address_and_port(instance)-- db_operation = DB_OPERATION_MAP.get(gen_ai_endpoint, "unknown")- if db_operation == "create_collection":- namespace = kwargs.get("name") or (args[0] if args else "unknown")- else:- namespace = getattr(instance, "name", "unknown") # For collection-based databases- span_name = f"{db_operation} {namespace}"-- with tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:- start_time = time.time()- response = wrapped(*args, **kwargs)-- try:- # Process response and generate telemetry- response = process_vectordb_response(- response, db_operation, server_address, server_port,- environment, application_name, metrics, start_time, span,- capture_message_content, disable_metrics, version,- instance, args, endpoint=gen_ai_endpoint, **kwargs- )-- except Exception as e:- handle_exception(span, e)-- return response-- return wrapper-```--### 3. Simplified Telemetry Pattern--Based on ChromaDB learnings, use a simplified, consistent pattern:--```python-def common_vectordb_logic(scope, environment, application_name,- metrics, capture_message_content, disable_metrics, version,- instance=None, endpoint=None):- """- Process vector database request and generate telemetry.- """- scope._end_time = time.time()-- # Set common database span attributes using helper- common_db_span_attributes(scope, SemanticConvention.DB_SYSTEM_{DATABASE_NAME},- scope._server_address, scope._server_port, environment, application_name, version)-- # Set DB operation specific attributes- scope._span.set_attribute(SemanticConvention.DB_OPERATION_NAME, scope._db_operation)- scope._span.set_attribute(SemanticConvention.DB_CLIENT_OPERATION_DURATION,- scope._end_time - scope._start_time)-- # Handle operations with endpoint differentiation- if scope._db_operation == SemanticConvention.DB_OPERATION_GET:- collection_name = getattr(instance, "name", "unknown")-- # Differentiate between different GET operations- if endpoint == "{database}.get":- query = scope._kwargs.get("ids", [])-- # Standard database attributes- scope._span.set_attribute(SemanticConvention.DB_QUERY_TEXT, str(query))- scope._span.set_attribute(SemanticConvention.DB_COLLECTION_NAME, collection_name)- scope._span.set_attribute(SemanticConvention.DB_VECTOR_COUNT, object_count(query))-- # Database-specific attributes- scope._span.set_attribute(SemanticConvention.DB_FILTER,- str(scope._kwargs.get("where", "")))-- # Response metrics- if scope._response and scope._response.get('ids'):- returned_rows = object_count(scope._response['ids']) # Adjust for response structure- scope._span.set_attribute(SemanticConvention.DB_RESPONSE_RETURNED_ROWS, returned_rows)-- scope._span.set_attribute(SemanticConvention.DB_QUERY_SUMMARY,- f"{scope._db_operation} {collection_name} "- f"ids={query} "- f"limit={scope._kwargs.get('limit', 'None')}")-- elif endpoint == "{database}.query":- # Handle query-specific parameters- query_texts = scope._kwargs.get("query_texts", [])- query_embeddings = scope._kwargs.get("query_embeddings", [])-- # Create query content representation- if query_texts:- query_content = f"texts={query_texts}"- elif query_embeddings:- query_content = f"embeddings={len(query_embeddings)} vectors"- else:- query_content = "no query provided"-- # Standard database attributes- scope._span.set_attribute(SemanticConvention.DB_QUERY_TEXT, query_content)- scope._span.set_attribute(SemanticConvention.DB_COLLECTION_NAME, collection_name)- scope._span.set_attribute(SemanticConvention.DB_VECTOR_QUERY_TOP_K,- scope._kwargs.get("n_results", 10))-- # Response metrics (adjust for nested response structure)- if scope._response and scope._response.get('ids'):- returned_rows = object_count(scope._response['ids'][0]) if scope._response['ids'] else 0- scope._span.set_attribute(SemanticConvention.DB_RESPONSE_RETURNED_ROWS, returned_rows)-- scope._span.set_attribute(SemanticConvention.DB_QUERY_SUMMARY,- f"{scope._db_operation} {collection_name} "- f"n_results={scope._kwargs.get('n_results', 10)} "- f"{query_content}")-- elif scope._db_operation == SemanticConvention.DB_OPERATION_INSERT:- collection_name = getattr(instance, "name", "unknown")- query = scope._kwargs.get("ids", [])-- # Standard database attributes- scope._span.set_attribute(SemanticConvention.DB_QUERY_TEXT, str(query))- scope._span.set_attribute(SemanticConvention.DB_COLLECTION_NAME, collection_name)- scope._span.set_attribute(SemanticConvention.DB_VECTOR_COUNT, object_count(query))-- scope._span.set_attribute(SemanticConvention.DB_QUERY_SUMMARY,- f"{scope._db_operation} {collection_name} "- f"ids={query} "- f"documents={scope._kwargs.get('documents', 'None')}")-- # ... similar patterns for other operations-- scope._span.set_status(Status(StatusCode.OK))-- # Record metrics using helper- if not disable_metrics:- record_db_metrics(metrics, SemanticConvention.DB_SYSTEM_{DATABASE_NAME},- scope._server_address, scope._server_port, environment, application_name,- scope._start_time, scope._end_time)-```--### 4. Key Telemetry Patterns--**Essential Attributes for All Operations:**-- `DB_QUERY_TEXT`: String representation of the query/operation-- `DB_COLLECTION_NAME`: Collection name (for collection-based databases)-- `DB_VECTOR_COUNT`: Count of primary query parameter-- `DB_QUERY_SUMMARY`: Comprehensive operation summary--**Query Summary Format:**-```python-f"{scope._db_operation} {collection_name} key_param={value} optional_param={value or 'None'}"-```--**Response Structure Handling:**-```python-# For flat arrays (like ChromaDB GET)-returned_rows = object_count(scope._response['ids'])--# For nested arrays (like ChromaDB QUERY)-returned_rows = object_count(scope._response['ids'][0]) if scope._response['ids'] else 0-```--## Naming Conventions--### 1. File Names-- Use lowercase with underscores: `{database_name}.py`-- Async files: `async_{database_name}.py` (only if needed)-- No hyphens, use underscores--### 2. Function Names-- Use `snake_case` consistently-- Standard function names:- - `general_wrap`- - `async_general_wrap` (if needed)- - `common_vectordb_logic`- - `process_vectordb_response`--### 3. Variable Names-- Use `snake_case` for all variables-- **Simplified naming**: Use `query` for primary operation parameter-- Standard variable names:- - `server_address`, `server_port`- - `start_time`, `end_time`- - `db_operation`- - `collection_name` (for collection-based databases)--### 4. Class Names-- Use `PascalCase` for classes-- Instrumentor class: `{DatabaseName}Instrumentor`--## Code Style Guidelines--### 1. Quotes-- **Always use double quotes** for strings: `"example"`-- **Never use single quotes** for strings: ❌ `'example'`-- Consistent across all files including inline comments, error messages, and string literals--### 2. Indentation-- **4 spaces** for indentation-- No tabs-- **Multi-line definitions**: Use **only one level** of indentation for continuation lines- ```python- # ✅ Correct: One level indentation for multi-line function definitions- def general_wrap(gen_ai_endpoint, version, environment, application_name,- tracer, pricing_info, capture_message_content, metrics, disable_metrics):-- # ✅ Correct: One level indentation for multi-line function calls- response = process_vectordb_response(- response, db_operation, server_address, server_port,- environment, application_name, metrics, start_time, span,- capture_message_content, disable_metrics, version,- instance, args, endpoint=gen_ai_endpoint, **kwargs- )-- # ❌ Wrong: Don't align with opening parenthesis- def general_wrap(gen_ai_endpoint, version, environment, application_name,- tracer, pricing_info, capture_message_content, metrics, disable_metrics):- ```--### 3. Comments-- **Triple double quotes** for docstrings:- ```python- def function_name():- """- Function description.- """- ```-- **Standard inline comments** for specific actions:- ```python- # Get server address and port using the standard helper- server_address, server_port = set_server_address_and_port(instance)-- # Process response and generate telemetry- response = process_vectordb_response(...)- ```--### 4. Import Order-Follow this specific order:-```python-# Standard library imports-import time--# OpenTelemetry imports-from opentelemetry.trace import Status, StatusCode, SpanKind-from opentelemetry import context as context_api--# OpenLIT imports-from openlit.__helpers import (- common_db_span_attributes,- record_db_metrics,- handle_exception,- set_server_address_and_port,-)-from openlit.semcov import SemanticConvention-```--### 5. Function Parameters-Standard parameter order:-```python-def function_name(gen_ai_endpoint, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics):-```--## Implementation Steps--### Step 1: Create Directory Structure-```bash-mkdir -p sdk/python/src/openlit/instrumentation/{database_name}-```--### Step 2: Create __init__.py (Compact Pattern)--**For databases with many operations, use the compact data structure approach:**--```python-"""-OpenLIT {DatabaseName} Instrumentation-"""--from typing import Collection-import importlib.metadata-from opentelemetry.instrumentation.instrumentor import BaseInstrumentor-from wrapt import wrap_function_wrapper--from openlit.instrumentation.{database_name}.{database_name} import general_wrap-# Only import async if the database supports it-# from openlit.instrumentation.{database_name}.async_{database_name} import async_general_wrap--_instruments = ("{database-package} >= {version}",)--# Operations to wrap for both sync and async clients-{DATABASE_NAME}_OPERATIONS = [- ("method_name", "{database}.endpoint"),- ("create_collection", "{database}.create_collection"),- ("upsert", "{database}.upsert"),- ("query", "{database}.query"),- ("delete", "{database}.delete"),- # ... add all operations here-]--class {DatabaseName}Instrumentor(BaseInstrumentor):- """- An instrumentor for {DatabaseName}'s client library.- """-- def instrumentation_dependencies(self) -> Collection[str]:- return _instruments-- def _instrument(self, **kwargs):- version = importlib.metadata.version("{database-package}")- environment = kwargs.get("environment", "default")- application_name = kwargs.get("application_name", "default")- tracer = kwargs.get("tracer")- pricing_info = kwargs.get("pricing_info", {})- capture_message_content = kwargs.get("capture_message_content", False)- metrics = kwargs.get("metrics_dict")- disable_metrics = kwargs.get("disable_metrics")-- # Wrap sync operations- for method_name, endpoint in {DATABASE_NAME}_OPERATIONS:- wrap_function_wrapper(- "{database_module}",- f"{ClientClass}.{method_name}",- general_wrap(- endpoint, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics- ),- )-- # Wrap async operations (only if database supports them)- # for method_name, endpoint in {DATABASE_NAME}_OPERATIONS:- # wrap_function_wrapper(- # "{database_module}",- # f"{AsyncClientClass}.{method_name}",- # async_general_wrap(- # endpoint, version, environment, application_name, tracer,- # pricing_info, capture_message_content, metrics, disable_metrics- # ),- # )-- def _uninstrument(self, **kwargs):- pass-```--**For databases with few operations, use the direct approach:**--```python-# Direct wrapping for databases with < 10 operations-wrap_function_wrapper(- "{database_module}",- "{Class}.{method}",- general_wrap(- "{database}.{operation}",- version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics- ),-)-```--### Step 3: Create Wrapper Files-Create sync (and async if needed) wrapper files following the general_wrap pattern.--### Step 4: Create utils.py-Include functions in **standard order**:-1. `DB_OPERATION_MAP` with proper semantic operation mapping-2. `object_count` helper function-3. `set_server_address_and_port` function for server address extraction-4. `common_vectordb_logic` with endpoint differentiation-5. `process_vectordb_response` function--**Server Address Extraction Function:**-```python-def set_server_address_and_port(instance):- """- Extracts server address and port from {database} client instance.-- Args:- instance: {Database} client instance-- Returns:- tuple: (server_address, server_port)- """- server_address = "default-host"- server_port = default_port-- # Database-specific extraction logic- # For Qdrant: instance.init_options- # For others: instance._client.base_url, instance.config.host, etc.-- return server_address, server_port-```--**Usage in wrapper files:**-```python-# Replace generic helper with database-specific function-from openlit.instrumentation.{database}.utils import set_server_address_and_port--# In wrapper function-server_address, server_port = set_server_address_and_port(instance)-```--### Step 5: Database-Specific Adaptations--**Server Address Patterns:**-- Pinecone: `"pinecone.io", 443`-- ChromaDB: `"localhost", 8000`-- Adjust for your database's default--**Collection vs Namespace:**-- Collection-based (ChromaDB): Use `DB_COLLECTION_NAME`-- Namespace-based (Pinecone): Use `DB_NAMESPACE`--**Response Structure:**-- Flat arrays: `object_count(response['ids'])`-- Nested arrays: `object_count(response['ids'][0])`--## Testing--### Create Test File-Create `sdk/python/tests/test_{database_name}.py`:--```python-"""-Tests for {DatabaseName} functionality.-"""--import pytest-import openlit-from {database_package} import {ClientClass}--# Initialize client-client = {ClientClass}()--# Initialize OpenLIT-openlit.init(environment="openlit-python-testing", application_name="openlit-python-{database}-test")--def test_sync_{database}_operation():- """- Tests synchronous operation.- """- try:- response = client.operation(- # parameters- )- assert response # Add appropriate assertions-- except Exception as e:- if "rate limit" in str(e).lower():- print("Rate limit exceeded:", e)- else:- raise--# Add async tests only if database supports async-# @pytest.mark.asyncio-# async def test_async_{database}_operation():-# """-# Tests asynchronous operation.-# """-# pass-```--## Integration--### Step 1: Add to Base Instrumentation-Add to `sdk/python/src/openlit/instrumentation/__init__.py`:--```python-from openlit.instrumentation.{database_name} import {DatabaseName}Instrumentor--def auto_instrument(**kwargs):- """- Auto-instrument supported libraries- """- # ... existing instrumentors ...-- # {DatabaseName} Instrumentation- try:- {DatabaseName}Instrumentor().instrument(**kwargs)- except Exception as e:- logger.warning(f"Failed to instrument {DatabaseName}: {e}")-```--### Step 2: Add Semantic Conventions-Add any new semantic conventions to `sdk/python/src/openlit/semcov/__init__.py`.--## Consistency Requirements--**CRITICAL**: All vector database instrumentations must follow **identical patterns** for maintainability and developer experience:--### **Code Style Consistency:**-- **Quotes**: Always use double quotes (`"`) - never single quotes (`'`)-- **Comments**: Use standardized inline comments (see Code Style Guidelines)-- **Function Order**: Follow the exact order specified in utils.py structure-- **Import Order**: Standard library → OpenTelemetry → OpenLIT imports-- **Variable Names**: Use `snake_case` consistently--### **Structural Consistency:**-- **Exception Handling**: `wrapped()` call outside try block, telemetry processing inside-- **Server Address**: Use `set_server_address_and_port(instance)` function signature-- **Span Creation**: Always use `tracer.start_as_current_span()` - never `tracer.start_span()`-- **Suppression Check**: Always include `context_api._SUPPRESS_INSTRUMENTATION_KEY` check--## Key Reminders--1. **Always use `tracer.start_as_current_span()`** - Never use `tracer.start_span()`-2. **Include suppression check** - Always check `context_api._SUPPRESS_INSTRUMENTATION_KEY`-3. **Use simplified patterns** - Focus on essential telemetry attributes-4. **Handle endpoint differentiation** - Use `endpoint` parameter for operations mapping to same semantic type-5. **Semantic operation mapping** - Multiple endpoints can map to same DB operation-6. **Response structure awareness** - Handle flat vs nested response arrays correctly-7. **Collection vs namespace** - Use appropriate semantic convention for your database type-8. **Default values** - Use `"None"` for missing values in query summaries (double quotes)-9. **Test thoroughly** - Include both sync and async tests (if applicable)--## Example Implementation--For complete reference implementations, see:--### **Qdrant** (Optimized Pattern):-- **Structure**: 4-file structure with async support-- **Lines**: 80 (__init__.py), 55 (sync), 55 (async), 328 (utils.py)-- **Features**: Compact __init__.py with data structure, `set_server_address_and_port` function-- **Pattern**: Collection-based, endpoint differentiation, comprehensive operation coverage--### **Pinecone** (Optimized Pattern):-- **Structure**: 4-file structure with async support-- **Lines**: 78 (__init__.py), 56 (sync), 56 (async), 227 (utils.py)-- **Features**: Compact __init__.py with data structure, `set_server_address_and_port` function-- **Pattern**: Namespace-based, comprehensive telemetry--### **ChromaDB** (Simplified Pattern):-- **Structure**: 3-file structure (no async support)-- **Lines**: 90 (__init__.py), 57 (sync), 273 (utils.py)-- **Features**: Collection-based, endpoint differentiation, `set_server_address_and_port` function-- **Pattern**: Direct wrapping approach for fewer operations--### **Recommended Approach**:-1. **< 10 operations**: Use ChromaDB pattern (direct wrapping)-2. **10-20 operations**: Use Pinecone pattern (traditional __init__.py)-3. **20+ operations**: Use Qdrant pattern (compact data structure)--This guide ensures consistency across all vector database instrumentations while accommodating database-specific requirements and maintaining high quality standards.\ No newline at end of filediff --git a/contributors/framework-instrumentation-guide.md b/contributors/framework-instrumentation-guide.mddeleted file mode 100644index fb2509cd..00000000--- a/contributors/framework-instrumentation-guide.md+++ /dev/null@@ -1,1421 +0,0 @@-# Framework Instrumentation Guide for OpenLIT--This guide provides a comprehensive, step-by-step process for adding new framework instrumentations or updating existing ones in OpenLIT. It's based on lessons learned from successful instrumentations including OpenAI Agents, Haystack, and others, ensuring consistency, performance, and competitive advantages across all framework integrations.--## Table of Contents--1. [Prerequisites](#prerequisites)-2. [Phase 1: Competitive Analysis](#phase-1-competitive-analysis)-3. [Phase 2: Framework Analysis](#phase-2-framework-analysis)-4. [Phase 3: Implementation Strategy](#phase-3-implementation-strategy)-5. [Phase 4: Code Implementation](#phase-4-code-implementation)-6. [Phase 5: Testing & Validation](#phase-5-testing--validation)-7. [Phase 6: Optimization](#phase-6-optimization)-8. [Phase 7: Post-Change Cleanup](#phase-7-post-change-cleanup)-9. [Code Standards & Patterns](#code-standards--patterns)-10. [Quality Checklist](#quality-checklist)--## Prerequisites--Before starting, ensure you have:-- OpenLIT development environment set up-- Target framework installed and working examples-- Access to competitor repositories for analysis-- Understanding of OpenTelemetry concepts--**CRITICAL**: Always use the venv in the root directory and test with test.py:-```bash-# ALWAYS use the venv in project root-source venv/bin/activate--# ALWAYS name your test file test.py (mandatory naming convention)-python test.py--# Alternative approach with explicit sys.path (for test.py):-sys.path.insert(0, "sdk/python/src")-```--**MANDATORY Testing Standards:**-- Test file must be named `test.py` (not test_framework.py or similar)-- Always use `source venv/bin/activate` first-- Place test.py in project root directory-- Use sys.path.insert(0, "sdk/python/src") in test.py--## Phase 1: Competitive Analysis--### Step 1.1: Clone and Research Competitor Implementations--**Primary Competitors:**-- [OpenInference](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation)-- [OpenLLMetry](https://github.com/traceloop/openllmetry/tree/main/packages)-- [AgentOps](https://github.com/AgentOps-AI/agentops)-- [LangFuse](https://github.com/langfuse/langfuse) (Not OpenTelemetry native)-- [LangSmith](https://github.com/langchain-ai/langsmith-sdk)--**Research Process:**-```bash-# Clone competitor repositories for deep analysis-mkdir competitive_analysis && cd competitive_analysis-git clone https://github.com/Arize-ai/openinference.git-git clone https://github.com/traceloop/openllmetry.git-git clone https://github.com/AgentOps-AI/agentops.git-git clone https://github.com/langfuse/langfuse.git-git clone https://github.com/langchain-ai/langsmith-sdk.git--# Navigate to framework-specific instrumentations-cd openinference/python/instrumentation/openinference-instrumentation-{framework}-cd openllmetry/packages/openllmetry-instrumentation-{framework}-```--**Deep Analysis Checklist:**-- [ ] **Integration Pattern**: Function wrapping vs native integration (like TracingProcessor)-- [ ] **Span Structure**: How many spans do they create and why?-- [ ] **🚨 CRITICAL: Span Hierarchy**: Do they maintain proper parent-child relationships? (MOST IMPORTANT)-- [ ] **Threading Context**: How do they handle ThreadPoolExecutor and async contexts?-- [ ] **Span Naming**: What naming convention do they use?-- [ ] **Attributes**: What attributes do they capture? Check against semantic conventions-- [ ] **Content Capture**: Do they capture input/output content with MIME types?-- [ ] **Business Intelligence**: Do they track cost, tokens, performance metrics?-- [ ] **Error Handling**: How do they handle framework version differences?-- [ ] **Performance**: How much overhead do they add?-- [ ] **Coverage**: Which framework operations do they instrument?--**⚠️ SPAN HIERARCHY IS CRITICAL**: This is the #1 issue that breaks observability. Frameworks often use ThreadPoolExecutor or async operations that break OpenTelemetry context propagation, resulting in "all root spans" instead of proper parent-child relationships.--### Step 1.2: Document Competitive Gaps and OpenLIT Advantages--Create detailed comparison:--| Feature | OpenInference | OpenLLMetry | OpenLIT Target |-|---------|---------------|-------------|----------------|-| Integration Method | Function wrapping | Function wrapping | Native (if available) |-| Span Count | X spans | Y spans | Z+ spans (comprehensive) |-| Business Intelligence | None/Basic | None/Basic | **Complete** (cost, tokens, metrics) |-| Content Capture | Basic | Basic | **Enhanced** (MIME types, structured) |-| Cross-System Tracing | Limited | Limited | **Full** (links to LLM providers) |-| Error Resilience | Basic | Basic | **Robust** (graceful degradation) |-| Semantic Conventions | Standard | Standard | **Extended** (custom attributes) |--**Key Insight**: OpenLIT's competitive advantage is **business intelligence** and **enhanced observability**.--### Step 1.2: Instrumentation Structure Patterns--**FLEXIBLE file structures based on framework complexity:**--```python-# PATTERN 1: Single File (Simple frameworks)-# Example: Simple utilities, basic wrappers-framework_name/-├── __init__.py # Instrumentation setup + all logic-└── (optional utils if needed)--# PATTERN 2: Processor Pattern (OpenAI Agents style)-# Example: Pipeline/processing-based frameworks-framework_name/-├── __init__.py # Instrumentation setup-└── processor.py # All processing logic, context handling--# PATTERN 3: Callback Pattern (LangChain style)-# Example: Frameworks with built-in callback systems-framework_name/-├── __init__.py # Instrumentation setup-└── callback_handler.py # Framework's callback interface--# PATTERN 4: RECOMMENDED 4-File Structure (Mem0, CrewAI, Pydantic AI)-# Example: Complex frameworks needing performance optimization-framework_name/-├── __init__.py # Instrumentation setup with separated SYNC/ASYNC methods-├── framework_name.py # Sync wrappers with threading context fixes-├── async_framework_name.py # Async wrappers-└── utils.py # Shared utilities with context caching and __slots__--**🎯 PREFERRED STRUCTURE**: Use Pattern 4 for any framework with threading issues or performance needs.--```--### Step 1.3: Threading Context Propagation Issues (CRITICAL)--**🚨 MOST COMMON FAILURE**: Frameworks using ThreadPoolExecutor break OpenTelemetry context propagation.--**Example Problem (like mem0):**-```python-# mem0/memory/main.py - Line 257-261-with concurrent.futures.ThreadPoolExecutor() as executor:- future1 = executor.submit(self._add_to_vector_store, messages, ...)- future2 = executor.submit(self._add_to_graph, messages, ...)- # ❌ This breaks context propagation - results in "all root spans"-```--**OpenLIT Solution Pattern:**-```python-# In utils.py - Threading context fix-def patch_concurrent_futures_context(span_context):- """Patch ThreadPoolExecutor to propagate OpenTelemetry context."""- original_submit = concurrent.futures.ThreadPoolExecutor.submit-- def patched_submit(self, fn, *args, **kwargs):- # Capture current context and propagate it- current_context = context.get_current()-- def context_wrapper(*args, **kwargs):- token = context.attach(span_context)- try:- return fn(*args, **kwargs)- finally:- context.detach(token)-- return original_submit(self, context_wrapper, *args, **kwargs)-- concurrent.futures.ThreadPoolExecutor.submit = patched_submit- return lambda: setattr(concurrent.futures.ThreadPoolExecutor, "submit", original_submit)--# In framework_name.py - Apply threading fix-def framework_wrap(gen_ai_endpoint, ...):- def wrapper(wrapped, instance, args, kwargs):- with tracer.start_as_current_span(gen_ai_endpoint, kind=span_kind, context=current_context) as span:- span_context = set_span_in_context(span, context=current_context)-- # ✅ Apply threading context fix for CLIENT operations- if span_kind == SpanKind.CLIENT:- restore_patch = patch_concurrent_futures_context(span_context)- try:- response = wrapped(*args, **kwargs)- finally:- restore_patch()- else:- response = wrapped(*args, **kwargs)-```--**🔧 When to Apply Threading Fixes:**-- Framework uses `concurrent.futures.ThreadPoolExecutor`-- Framework uses `asyncio.run_in_executor`-- You see "all root spans" in your hierarchy test-- Child operations (OpenAI, Qdrant, etc.) appear as separate root spans--### Step 1.4: Performance Optimization Patterns--**CRITICAL**: Study existing OpenLIT implementations for optimization patterns:--```python-# Study these proven patterns in existing instrumentations:-# - Mem0: threading context propagation, __slots__ optimization-# - CrewAI: excellent agent patterns, lifecycle management-# - OpenAI Agents: great tool handling (processor.py pattern)-# - LangChain/LlamaIndex: mature framework patterns, utils.py caching-# - Pydantic AI: context caching with PydanticAIInstrumentationContext--# Example: Optimized context caching with __slots__-class FrameworkInstrumentationContext:- """Context object to cache expensive extractions."""-- __slots__ = ("instance", "args", "kwargs", "version", "environment",- "application_name", "_agent_name", "_model_name", "_tools", "_messages")-- def __init__(self, instance, args, kwargs, version, environment, application_name):- self.instance = instance- self.args = args- self.kwargs = kwargs- self.version = version- self.environment = environment- self.application_name = application_name-- # Cache expensive operations with lazy loading- self._agent_name = None- self._model_name = None- self._tools = None- self._messages = None-- @property- def agent_name(self) -> str:- """Lazy-loaded with caching."""- if self._agent_name is None:- self._agent_name = self._extract_agent_name()- return self._agent_name-```--## Phase 2: Framework Analysis--### Step 2.1: Explore Framework's Built-in Capabilities--**CRITICAL**: Before implementing, check if the framework has built-in tracing/monitoring:--```python-# Example: Discover OpenAI Agents has native tracing-import {framework}--# Look for tracing, monitoring, observability features-# Check documentation for:-# - Built-in tracing systems-# - Processor patterns-# - Event hooks-# - Monitoring callbacks--# Example: OpenAI Agents has TracingProcessor-from agents import TracingProcessor, set_trace_processors-```--**Framework Integration Decision Tree:**-1. **Native Integration Available** (e.g., OpenAI Agents TracingProcessor)- - ✅ Use native integration for perfect hierarchy- - ✅ Extends framework's built-in system- - ✅ Better performance and reliability--2. **Function Wrapping Required** (most frameworks)- - ✅ Standard approach for frameworks without native tracing- - ⚠️ Requires careful hierarchy management--### Step 2.2: Clone and Study Target SDK (CRITICAL)--**ALWAYS clone the actual SDK you're instrumenting:**--### Step 2.3: Semantic Conventions Are CRITICAL--**NEVER hardcode strings - ALWAYS use SemanticConvention:**--```python-# ❌ WRONG - will be rejected in code review:-span.set_attribute("gen_ai.operation.name", "chat")-span.set_attribute("gen_ai.system", "pydantic_ai")--# ✅ CORRECT - must use semantic conventions:-from openlit.semcov import SemanticConvention--span.set_attribute(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT)-span.set_attribute(SemanticConvention.GEN_AI_PROVIDER_NAME, SemanticConvention.GEN_AI_SYSTEM_PYDANTIC_AI)--# Check existing constants first:-# - GEN_AI_OPERATION_TYPE_*-# - GEN_AI_AGENT_LIFECYCLE_PHASE_*-# - GEN_AI_SYSTEM_* (add new ones if needed)--# If needed, add to semcov.py:-class SemanticConvention:- # Add framework-specific constants- GEN_AI_SYSTEM_PYDANTIC_AI = "pydantic_ai"- GEN_AI_AGENT_LIFECYCLE_PHASE_GRAPH_EXECUTION = "graph_execution"-```--### Step 2.4: Map Framework Operations--**Workflow vs Component Operations:**-```python-# Example for any framework-WORKFLOW_OPERATIONS = [- # High-level operations users care about in production- "agent.run_sync", # OpenAI Agents- "pipeline.run", # Haystack- "workflow.execute" # Generic-]--COMPONENT_OPERATIONS = [- # Detailed operations for debugging (detailed_tracing=True)- "retriever.retrieve",- "generator.generate",- "embedder.embed",- "tool.call"-]-```--### Step 2.3: Test Framework Methods and Patterns--**Method Discovery Script:**-```python-# test_framework_exploration.py-import {framework}--# Discover available methods and attributes-obj = SomeFrameworkClass()-print("Available methods:", [m for m in dir(obj) if not m.startswith('_')])--# Test built-in monitoring-if hasattr(obj, 'trace') or hasattr(obj, 'monitor'):- print("Built-in tracing available!")--# Check for callback patterns-if hasattr(obj, 'add_callback') or hasattr(obj, 'on_event'):- print("Event system available!")-```--## Phase 3: Implementation Strategy--### Step 3.1: Choose Integration Pattern--**Pattern 1: Native Integration (Preferred when available)**-```python-# Example: OpenAI Agents TracingProcessor-class OpenLITTracingProcessor(TracingProcessor):- def on_trace_start(self, trace): pass- def on_span_start(self, span): pass- def on_span_end(self, span): pass-```--**Pattern 2: Function Wrapping (Standard)**-```python-# Traditional OpenLIT pattern-wrap_function_wrapper("module", "Class.method", wrapper)-```--### Step 3.2: Plan Span Naming Convention--**CRITICAL**: Follow consistent naming pattern:-```python-# Standard format: "{operation_type} {operation_name}"-"agent Triage agent" # operation_type=agent, operation_name=Triage agent-"chat gpt-4o" # operation_type=chat, operation_name=gpt-4o-"retrieve documents" # operation_type=retrieve, operation_name=documents-"generate response" # operation_type=generate, operation_name=response-```--### Step 3.3: Plan Semantic Conventions Usage--**Extend semcov.py when needed:**-```python-# It's OK to add new semantic conventions for better attributes-class SemanticConvention:- # Add framework-specific conventions- GEN_AI_AGENT_DESCRIPTION = "gen_ai.agent.description"- GEN_AI_AGENT_VERSION = "gen_ai.agent.version"- GEN_AI_WORKFLOW_TYPE = "gen_ai.workflow.type"-```--## Phase 4: Code Implementation--### Step 4.1: Create Span Hierarchy Test First (MANDATORY)--**🚨 CRITICAL**: Create test.py (mandatory name) with span hierarchy analysis BEFORE any implementation:--```python-# test.py - MANDATORY filename, must be in project root-#!/usr/bin/env python3-"""-MANDATORY span hierarchy test for framework instrumentation.-This MUST be created first and MUST verify proper parent-child relationships.-"""--import sys-import os--sys.path.insert(0, "sdk/python/src") # MANDATORY path setup--import openlit-import logging--# Minimal logging to focus on span hierarchy-logging.getLogger().setLevel(logging.ERROR)--def test_framework_hierarchy():- """- 🚨 CRITICAL TEST: Verify span hierarchy is correct, not "all root spans"- """- print("🧠 Framework Instrumentation Test")- print("=" * 40)-- # Initialize OpenLIT - use venv: source venv/bin/activate- print("✅ Initializing OpenLIT...")- openlit.init(detailed_tracing=True)-- # Import and use framework- print("✅ Testing framework operations...")- from your_framework import YourClass-- instance = YourClass()-- # Test operations that should create proper hierarchy- result1 = instance.main_operation("test data", user_id="test_user")- print(f"✅ Main operation result: {result1}")-- result2 = instance.secondary_operation("search query", user_id="test_user", limit=1)- print(f"✅ Secondary operation result: {len(result2) if result2 else 0} items")-- print("\n🎯 EXPECTED HIERARCHY:")- print("🔸 ROOT: main_operation")- print(" ↳ internal_operation_1")- print(" ↳ openai_call (or similar)")- print(" ↳ database_operation")- print("🔸 ROOT: secondary_operation")- print(" ↳ internal_operation_2")- print(" ↳ embedding_call")-- print("\n⚠️ VERIFY: Check console output above for proper parent-child relationships")- print("❌ If you see 'all root spans', you have threading context propagation issues!")--if __name__ == "__main__":- test_framework_hierarchy()-```--**🔧 SPAN HIERARCHY DEBUGGING COMMANDS:**--```bash-# Run test.py and analyze hierarchy-source venv/bin/activate # MANDATORY first step-python test.py > spans.json 2>&1--# Parse and analyze hierarchy (create this script)-python3 -c "-import json-# ... hierarchy parsing script from mem0 experience ...-# Shows proper tree structure or identifies 'all root spans' issue-"-```--**⚠️ HIERARCHY TESTING IS THE MOST IMPORTANT STEP**:-- If hierarchy is broken (all root spans), nothing else matters-- This is the #1 failure mode in framework instrumentation-- Threading context propagation issues MUST be caught early--### Step 4.2: Implement Based on Integration Pattern--**For Native Integration:**-```python-# processor.py - When framework has built-in tracing-class OpenLITTracingProcessor(FrameworkProcessor):- def on_span_start(self, span):- # Create OpenTelemetry span with proper naming- span_name = self._get_span_name(span) # {operation_type} {operation_name}- otel_span = self._tracer.start_span(span_name, kind=SpanKind.CLIENT)-- # Set semantic convention attributes- self._set_common_attributes(otel_span, span)-- def _get_span_name(self, span):- operation_type = self._get_operation_type(span.span_data)- operation_name = self._extract_operation_name(span.span_data)- return f"{operation_type} {operation_name}"-```--**For Function Wrapping:**-```python-# Follow existing 4-file structure-# __init__.py, sync_wrapper.py, async_wrapper.py, utils.py-```--### Step 4.3: Implement Comprehensive Attributes--**Use semantic conventions extensively:**-```python-def _set_span_attributes(self, span, data):- # Standard framework attributes- span.set_attribute(SemanticConvention.GEN_AI_PROVIDER_NAME, "framework_name")- span.set_attribute(SemanticConvention.GEN_AI_OPERATION_NAME, operation_name)-- # Model information (critical for business intelligence)- model = self._extract_model_info(data)- if model:- span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, model)-- # Content capture with MIME types (OpenLIT enhancement)- if self._capture_message_content:- self._capture_input_output_with_mime_types(span, data)-- # Business intelligence attributes- self._capture_token_usage(span, data)- self._capture_cost_metrics(span, data)-- # Framework-specific attributes- if hasattr(data, 'agent_name'):- span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, data.agent_name)-```--## Phase 5: Testing & Validation--### Step 5.1: Create Comprehensive Test Suite--**Test Categories:**-1. **Span Hierarchy Test** (most important)-2. **Competitive Comparison Test** (CRITICAL)-3. **Performance Benchmarking Test**-4. **Mock Framework Test** (when real framework unavailable)-5. **Error Resilience Test**--**Span Hierarchy Validation:**-```python-def test_span_hierarchy():- """Validate proper parent-child relationships and span naming"""- spans = run_instrumentation_test()-- # Validate hierarchy structure- assert len(spans) > 0, "Must generate spans"-- # Check root span exists- root_spans = [s for s in spans if s['parent_id'] is None]- assert len(root_spans) == 1, "Must have exactly one root span"-- # Validate span naming convention- for span in spans:- assert ' ' in span['name'], f"Span name '{span['name']}' should follow '{{operation_type}} {{operation_name}}' format"-- # Check semantic conventions- for span in spans:- attrs = span['attributes']- assert 'gen_ai.system' in attrs, "Must have gen_ai.system attribute"-```--### Step 5.2: Competitive Validation--**Critical Test - OpenLIT vs Competitors:**-```python-def test_competitive_superiority():- """Validate OpenLIT generates more/better spans than competitors"""-- # Test OpenLIT- openlit_spans = test_openlit_instrumentation()-- # Test competitors (with error handling)- try:- competitor1_spans = test_competitor_instrumentation("openinference")- competitor2_spans = test_competitor_instrumentation("openllmetry")- competitor3_spans = test_competitor_instrumentation("agentops")- except Exception as e:- print(f"Competitor failed: {e}")- competitor1_spans = competitor2_spans = competitor3_spans = 0-- # Validate OpenLIT superiority- assert openlit_spans > 0, "OpenLIT must generate spans"- print(f"OpenLIT: {openlit_spans} spans (with business intelligence)")- print(f"OpenInference: {competitor1_spans} spans")- print(f"OpenLLMetry: {competitor2_spans} spans")- print(f"AgentOps: {competitor3_spans} spans")-- # OpenLIT should provide comprehensive coverage- assert openlit_spans >= max(competitor1_spans, competitor2_spans, competitor3_spans)-```--### Step 5.3: Performance Benchmarking--**Critical Test - Performance vs Competitors:**-```python-def benchmark_instrumentation_performance():- """Compare instrumentation overhead across competitors."""-- import time-- # Test scenarios- test_cases = [- "single_agent_simple_task",- "multi_agent_conversation",- "tool_heavy_workflow",- "high_frequency_requests"- ]-- competitors = ["openlit", "openinference", "openllmetry", "agentops"]-- results = {}- for competitor in competitors:- results[competitor] = {}-- for test_case in test_cases:- # Baseline without instrumentation- start = time.time()- run_test_case(test_case, instrumentation=None)- baseline = time.time() - start-- # With competitor instrumentation- start = time.time()- run_test_case(test_case, instrumentation=competitor)- instrumented = time.time() - start-- overhead = ((instrumented - baseline) / baseline) * 100- results[competitor][test_case] = {- "overhead_percent": overhead,- "baseline_ms": baseline * 1000,- "instrumented_ms": instrumented * 1000- }-- # Validate OpenLIT performance- openlit_avg_overhead = sum(results["openlit"][tc]["overhead_percent"] for tc in test_cases) / len(test_cases)- assert openlit_avg_overhead < 10, f"OpenLIT overhead too high: {openlit_avg_overhead:.2f}%"-- return results-```--### Step 5.3: Mock Testing When Framework Unavailable--**Create realistic mock scenarios:**-```python-def test_with_mocks():- """Test instrumentation when real framework isn't available"""- tracer = trace.get_tracer(__name__)-- # Create realistic span hierarchy manually- with tracer.start_as_current_span("agent Agent workflow") as root:- root.set_attribute("gen_ai.agent.name", "Agent workflow")-- with tracer.start_as_current_span("agent Triage agent") as agent:- agent.set_attribute("gen_ai.agent.name", "Triage agent")-- with tracer.start_as_current_span("chat gpt-4o") as chat:- chat.set_attribute("gen_ai.request.model", "gpt-4o")-```--## Phase 6: Optimization--### Step 6.1: Context Caching for Performance--**Implement context caching pattern (learned from Pydantic AI optimization):**--```python-class FrameworkInstrumentationContext:- """Context object to cache expensive extractions and reduce performance overhead."""-- def __init__(self, instance, args, kwargs, version, environment, application_name):- self.instance = instance- self.args = args- self.kwargs = kwargs- self.version = version- self.environment = environment- self.application_name = application_name-- # Cache expensive operations with lazy loading- self._agent_name = None- self._model_name = None- self._server_info = None- self._messages = None- self._tools = None- self._model_params = None-- @property- def agent_name(self) -> str:- """Get agent name with caching - avoids repeated extraction."""- if self._agent_name is None:- self._agent_name = getattr(self.instance, 'name', None) or "default_agent"- return self._agent_name-- @property- def model_name(self) -> str:- """Get model name with caching."""- if self._model_name is None:- if hasattr(self.instance, 'model') and hasattr(self.instance.model, 'model_name'):- self._model_name = str(self.instance.model.model_name)- else:- self._model_name = "unknown"- return self._model_name--def set_span_attributes(span, operation_name: str, ctx: FrameworkInstrumentationContext,- lifecycle_phase: Optional[str] = None,- additional_attrs: Optional[Dict[str, Any]] = None):- """Optimized attribute setting with context caching."""-- # Set core attributes using cached context- span.set_attribute(SemanticConvention.GEN_AI_OPERATION, operation_name)- span.set_attribute(SemanticConvention.GEN_AI_PROVIDER_NAME, SemanticConvention.GEN_AI_SYSTEM_FRAMEWORK)- span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, ctx.agent_name)- span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, ctx.model_name)-- # Set environment attributes- span.set_attribute(DEPLOYMENT_ENVIRONMENT, ctx.environment)- span.set_attribute(SERVICE_NAME, ctx.application_name)- span.set_attribute(SemanticConvention.GEN_AI_SDK_VERSION, ctx.version)-- # Set lifecycle phase if provided- if lifecycle_phase:- span.set_attribute(SemanticConvention.GEN_AI_AGENT_LIFECYCLE_PHASE, lifecycle_phase)-- # Set additional attributes- if additional_attrs:- for key, value in additional_attrs.items():- span.set_attribute(key, value)-```--### Step 6.2: Performance Validation--**Measure instrumentation overhead:**-```python-def benchmark_performance():- """Measure instrumentation performance impact"""- import time-- # Test without instrumentation- start = time.time()- run_framework_operations(count=100)- baseline = time.time() - start-- # Test with instrumentation- openlit.init()- start = time.time()- run_framework_operations(count=100)- instrumented = time.time() - start-- overhead = ((instrumented - baseline) / baseline) * 100- print(f"Instrumentation overhead: {overhead:.2f}%")- assert overhead < 10, "Overhead should be under 10%"-```--### Step 6.2: Business Intelligence Validation--**Verify OpenLIT's competitive advantages:**-```python-def test_business_intelligence():- """Validate comprehensive business metrics capture"""- spans = run_instrumentation_test()-- business_attributes = [- 'gen_ai.usage.input_tokens',- 'gen_ai.usage.output_tokens',- 'gen_ai.client.operation.duration',- 'gen_ai.request.model'- ]-- for span in spans:- attrs = span['attributes']- found_attrs = [attr for attr in business_attributes if attr in attrs]- print(f"Span '{span['name']}': {len(found_attrs)}/{len(business_attributes)} business attributes")-```--## Phase 7: Post-Change Cleanup--**CRITICAL**: Always run comprehensive cleanup after implementation:--### Step 7.1: Automated Code Quality Check--```bash-#!/bin/bash-# cleanup_instrumentation.sh-FRAMEWORK_DIR="src/openlit/instrumentation/$1"--echo "🧹 Cleaning up $FRAMEWORK_DIR..."--# Remove trailing whitespace-find "$FRAMEWORK_DIR" -name "*.py" -exec sed -i '' 's/[[:space:]]*$//' {} \;--# Add missing final newlines-find "$FRAMEWORK_DIR" -name "*.py" | while read file; do- if [[ ! -s "$file" || $(tail -c1 "$file" | wc -l) -eq 0 ]]; then- echo "" >> "$file"- fi-done--# Check syntax-for file in "$FRAMEWORK_DIR"/*.py; do- python3 -m py_compile "$file" || echo "❌ Syntax error in $file"-done--# Check line lengths-long_lines=$(find "$FRAMEWORK_DIR" -name "*.py" -exec awk 'length($0) > 80' {} \; | wc -l)-echo "Lines over 80 characters: $long_lines"-```--### Step 7.2: Comprehensive Pylint Error Handling--**🚨 MANDATORY: 10.00/10 PYLINT SCORE REQUIRED**--**Run Pylint Check with Project Configuration:**-```bash-# ALWAYS use venv first, then run from SDK directory-source venv/bin/activate-cd sdk/python-python -m pylint src/openlit/instrumentation/{framework}/ --rcfile=.pylintrc--# TARGET SCORE: 10.00/10 (not 9.5, must be perfect)-```--**Run Script for Perfect 10.0/10:**-```bash-# Use the script pattern from mem0 success-#!/bin/bash-cd /Users/user/openlit # Adjust path-source venv/bin/activate-export PYTHONPATH="/Users/user/openlit/sdk/python/src:$PYTHONPATH"-cd sdk/python-python -m pylint src/openlit/instrumentation/{framework}/ --rcfile=.pylintrc-```--**Note**: The project's `.pylintrc` disables many common warnings like `broad-exception-caught`, `too-many-locals`, etc.--### Step 7.2b: Pylint Optimization Patterns--**Common High-Impact Fixes:**--```python-# 1. Lazy logging (prevents W1203: logging-fstring-interpolation)-# ❌ BAD:-logger.debug(f"Failed to extract messages: {e}")--# ✅ GOOD:-logger.debug("Failed to extract messages: %s", e)--# 2. Remove unused imports and variables-# ❌ BAD:-import importlib.metadata-from typing import Dict, Any, Optional, List, Union, Tuple--def wrapper(wrapped, instance, args, kwargs):- method_name = wrapped.__name__ # Unused variable- return wrapped(*args, **kwargs)--# ✅ GOOD:-import json-from typing import Dict, Any, Optional, List, Tuple--def wrapper(wrapped, instance, args, kwargs):- return wrapped(*args, **kwargs)--# 3. Fix exception handling-# ❌ BAD:-try:- # some code-except Exception as e: # Unused variable- pass--# ✅ GOOD:-try:- # some code-except Exception:- pass--# 4. Add class methods for R0903: Too few public methods-# ❌ BAD:-class CreateContext:- def __init__(self):- self.data = {}--# ✅ GOOD:-class CreateContext:- """Context for agent creation instrumentation."""- def __init__(self):- self.data = {}-- def get_context_info(self):- """Get context information."""- return self.data-- def has_data(self):- """Check if context has data."""- return bool(self.data)-```--### Step 7.3: Configure Project Pylint Rules--**Add `import-outside-toplevel` to disabled rules in `sdk/python/.pylintrc`:**--```ini-[MESSAGES CONTROL]-disable=- # ... existing rules ...- import-outside-toplevel # Essential for optional dependencies-```--**Why**: Import-outside-toplevel warnings are triggered by correct optional dependency patterns in OpenLIT.--### Step 7.4: Optimize Dummy Classes for Perfect Score--**Eliminate unnecessary dummy classes to achieve 10.00/10:**--#### **Use TYPE_CHECKING Pattern for Type-Only Classes**-```python-from typing import TYPE_CHECKING--try:- from framework import MainClass- if TYPE_CHECKING:- from framework import TypeOnlyClass1, TypeOnlyClass2- AVAILABLE = True-except ImportError:- class MainClass:- """Dummy class - actually used at runtime"""- def method(self): return None-- if TYPE_CHECKING:- # Type hints only - don't exist at runtime- TypeOnlyClass1 = Any- TypeOnlyClass2 = Any- AVAILABLE = False--# Use quoted type hints for methods-def process(self, item: "TypeOnlyClass1") -> None:-```--#### **When R0903 is Still Acceptable**-- **Status**: ✅ **ACCEPTABLE** for classes actually used at runtime-- **Example**: TracingProcessor base class that's inherited from--**Target Score**: **10.00/10** is achievable for most instrumentations--### Step 7.5: Common Pylint Issues and Fixes--#### **C0301: Line Too Long**-**Problem**: Lines exceed character limit (135 characters per project pylintrc)--**Fix Pattern**:-```python-# BAD: Long function call-span.set_attribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, handle_not_given(kwargs.get("frequency_penalty"), 0.0))--# GOOD: Split across multiple lines-span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- handle_not_given(kwargs.get("frequency_penalty"), 0.0)-)--# BAD: Long function call with many parameters-common_span_attributes(scope, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT, SemanticConvention.GEN_AI_SYSTEM_OPENAI, server_address, server_port, request_model, response_model, environment, application_name, is_stream, tbt, ttft, version)--# GOOD: Split parameters logically-common_span_attributes(- scope,- SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- SemanticConvention.GEN_AI_SYSTEM_OPENAI,- server_address, server_port, request_model,- response_model, environment, application_name,- is_stream, tbt, ttft, version-)--# BAD: Long import-from openlit.instrumentation.framework.async_framework import async_general_wrap--# GOOD: Split import-from openlit.instrumentation.framework.async_framework import (- async_general_wrap-)-```--#### **C0303: Trailing Whitespace**-**Automated Fix**:-```bash-# Remove all trailing whitespace-find src/openlit/instrumentation/{framework} -name "*.py" -exec sed -i '' 's/[[:space:]]*$//' {} \;-```--#### **C0304: Missing Final Newline**-**Automated Fix**:-```bash-# Add missing final newlines-find src/openlit/instrumentation/{framework} -name "*.py" | while read file; do- if [[ ! -s "$file" || $(tail -c1 "$file" | wc -l) -eq 0 ]]; then- echo "" >> "$file"- fi-done-```--#### **C0115: Missing Class Docstring**-**Fix Pattern**:-```python-# BAD: No docstring-class TracingProcessor:- def force_flush(self): pass--# GOOD: Add descriptive docstring-class TracingProcessor:- """Dummy TracingProcessor class for when agents is not available"""-- def force_flush(self):- """Dummy force_flush method"""- pass-```--#### **C0116: Missing Function Docstring**-**Fix Pattern**:-```python-# BAD: No docstring-def _extract_model_info(self, data):- return data.model--# GOOD: Add descriptive docstring-def _extract_model_info(self, data):- """Extract model information from span data or agent configuration"""- return data.model-```--#### **C0321: Multiple Statements on Single Line**-**Fix Pattern**:-```python-# BAD: Multiple statements-def force_flush(self): pass-class Trace: pass--# GOOD: Separate lines with proper formatting-def force_flush(self):- """Dummy force_flush method"""- pass--class Trace:- """Dummy Trace class for when agents is not available"""- pass-```--#### **C0415: Import Outside Toplevel**-**Context**: These warnings are **ACCEPTABLE** and **REQUIRED** for optional dependencies--**Correct Pattern (DO NOT "FIX")**:-```python-# This is CORRECT for optional dependencies - keep as-is-def _instrument(self, **kwargs):- try:- from agents import set_trace_processors # CORRECT inside function- set_trace_processors([processor])- except ImportError:- pass # Package not available-```--**Why This is Correct**:-- Prevents import errors when target package isn't installed-- Essential for OpenLIT's optional dependency pattern-- Moving to top level would break instrumentation--**Only Fix When**:-```python-# ONLY move to top level for always-needed imports-import json # At top of file when always used-from typing import Any, Dict, Optional--def _capture_model_parameters(self, span, data):- try:- params = {"model": data.model}- span.set_attribute("gen_ai.request.parameters", json.dumps(params))- except Exception:- pass-```--#### **E0602: Undefined Variable**-**Problem**: Variable used before definition--**Fix Pattern**:-```python-# BAD: Undefined variable-cost = get_chat_model_cost(model, pricing_info, input_tokens, output_tokens)--# GOOD: Import the function-from openlit.__helpers import (- common_framework_span_attributes,- handle_exception,- get_chat_model_cost # Add missing import-)-```--#### **Syntax Errors: Missing except/finally**-**Problem**: Try block without proper exception handling--**Fix Pattern**:-```python-# BAD: Missing except block-def _extract_token_usage(self, span, data):- try:- usage = data.usage- span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens)--# GOOD: Add except block-def _extract_token_usage(self, span, data):- try:- usage = data.usage- span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens)- except Exception:- pass # Ignore errors in token usage extraction-```--#### **R1702: Too Many Nested Blocks**-**Fix Pattern**:-```python-# BAD: Deep nesting-def process_data(self, data):- if hasattr(data, 'config'):- if data.config:- if hasattr(data.config, 'model'):- if data.config.model:- if isinstance(data.config.model, str):- return data.config.model--# GOOD: Early returns-def process_data(self, data):- if not hasattr(data, 'config'):- return None- if not data.config:- return None- if not hasattr(data.config, 'model'):- return None- if not data.config.model:- return None- if not isinstance(data.config.model, str):- return None- return data.config.model-```--#### **R0903: Too Few Public Methods**-**Context**: **ACCEPTABLE** and **EXPECTED** for dummy/placeholder classes--**Correct Pattern (DO NOT "FIX")**:-```python-# This is CORRECT for dummy classes - keep as-is-class TracingProcessor:- """Dummy TracingProcessor class for when agents is not available"""-- def force_flush(self):- """Dummy force_flush method"""- return None-- def shutdown(self):- """Dummy shutdown method"""- return None-```--**Why This is Correct**:-- Dummy classes by design to prevent import errors-- Only need minimal methods to satisfy interfaces-- Adding unnecessary methods would be incorrect--### Step 7.6: Automated Pylint Fix Script--**Create comprehensive fix script:**-```bash-#!/bin/bash-# fix_pylint_issues.sh-FRAMEWORK_DIR="src/openlit/instrumentation/$1"--echo "🔧 Fixing Pylint issues in $FRAMEWORK_DIR..."--# 1. Remove trailing whitespace-echo " ✂️ Removing trailing whitespace..."-find "$FRAMEWORK_DIR" -name "*.py" -exec sed -i '' 's/[[:space:]]*$//' {} \;--# 2. Add missing final newlines-echo " 📝 Adding missing final newlines..."-find "$FRAMEWORK_DIR" -name "*.py" | while read file; do- if [[ ! -s "$file" || $(tail -c1 "$file" | wc -l) -eq 0 ]]; then- echo "" >> "$file"- fi-done--# 3. Check syntax errors-echo " 🔍 Checking syntax..."-syntax_errors=0-for file in "$FRAMEWORK_DIR"/*.py; do- if ! python3 -m py_compile "$file" 2>/dev/null; then- echo " ❌ Syntax error in $file"- syntax_errors=$((syntax_errors + 1))- fi-done--# 4. Check line lengths and provide guidance-echo " 📏 Checking line lengths..."-long_lines=$(find "$FRAMEWORK_DIR" -name "*.py" -exec awk 'length($0) > 135 { print FILENAME ":" NR ":" length($0) ":" $0 }' {} \;)-if [ -n "$long_lines" ]; then- echo " ⚠️ Long lines found (>135 chars):"- echo "$long_lines" | head -5- echo " 💡 Split long lines using the patterns in the guide"-fi--# 5. Final validation-echo " ✅ Running final validation..."-if [ $syntax_errors -eq 0 ]; then- echo "✅ All syntax errors fixed!"-else- echo "❌ $syntax_errors syntax errors remaining - manual fix required"-fi--# 6. Run limited pylint check-echo " 🔍 Running key pylint checks..."-pylint --disable=all --enable=syntax-error,undefined-variable,trailing-whitespace,missing-final-newline "$FRAMEWORK_DIR" 2>/dev/null || echo "Some pylint issues remain - check manually"--echo "🎉 Pylint fix script completed!"-echo "💡 Run full pylint check and fix remaining issues manually"-```--### Step 7.7: Manual Fix Checklist--**Before committing, verify:**-- [ ] ✅ All files compile without syntax errors-- [ ] ✅ No trailing whitespace-- [ ] ✅ All files end with newline-- [ ] ✅ No lines over 135 characters (or split appropriately)-- [ ] ✅ All classes have docstrings-- [ ] ✅ All public methods have docstrings-- [ ] ✅ No undefined variables-- [ ] ✅ Try blocks have except/finally-- [ ] ✅ Imports are at top level when possible-- [ ] ✅ Nested blocks reduced where possible--### Step 7.8: Usage Instructions--**Run the fix script:**-```bash-chmod +x fix_pylint_issues.sh-./fix_pylint_issues.sh openai_agents-./fix_pylint_issues.sh haystack-```--**Final validation:**-```bash-# Check specific framework-pylint src/openlit/instrumentation/openai_agents/--# Quick syntax check-python3 -m py_compile src/openlit/instrumentation/openai_agents/*.py--# Test instrumentation still works-cd sdk/python && PYTHONPATH=src python -c "-import openlit-openlit.init()-print('✅ Instrumentation works after pylint fixes')-"-```--### Step 7.9: Testing After Pylint Fixes--**CRITICAL**: Always test that instrumentation works after fixing pylint issues:--```python-# test_after_pylint_fix.py-import sys-sys.path.insert(0, 'sdk/python/src')--import openlit-from opentelemetry.sdk.trace import TracerProvider-from opentelemetry.sdk.trace.export import SimpleSpanProcessor--class TestExporter:- def __init__(self):- self.spans = []- def export(self, spans):- self.spans.extend(spans)- return 0- def shutdown(self):- pass--def test_instrumentation_after_fixes():- """Test that instrumentation works after pylint fixes"""- exporter = TestExporter()- tracer_provider = TracerProvider()- tracer_provider.add_span_processor(SimpleSpanProcessor(exporter))-- # Initialize OpenLIT- openlit.init(detailed_tracing=True)-- # Run basic test based on framework- # ... framework-specific test code ...-- tracer_provider.force_flush(1000)-- assert len(exporter.spans) > 0, "Instrumentation should generate spans"- print(f"✅ Generated {len(exporter.spans)} spans after pylint fixes")-- # Validate span naming convention- for span in exporter.spans:- assert ' ' in span.name, f"Span '{span.name}' should follow '{{operation_type}} {{operation_name}}' format"-- print("✅ All tests passed - instrumentation working correctly")--if __name__ == "__main__":- test_instrumentation_after_fixes()-```--## Code Standards & Patterns--### Span Naming Convention--**MANDATORY Format**: `{operation_type} {operation_name}`--```python-# ✅ CORRECT-"agent Triage agent"-"chat gpt-4o"-"retrieve documents"-"generate response"-"workflow multi-agent"--# ❌ INCORRECT-"Triage agent" # Missing operation type-"agent_execution" # Wrong format-"chat_completion_gpt4" # Wrong format-```--### Semantic Conventions Usage--**Extensive usage required - add to semcov.py when needed:**--```python-# Always use semantic conventions-span.set_attribute(SemanticConvention.GEN_AI_PROVIDER_NAME, "framework_name")-span.set_attribute(SemanticConvention.GEN_AI_OPERATION_NAME, operation_name)-span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, model)--# Add new conventions when needed-class SemanticConvention:- GEN_AI_AGENT_DESCRIPTION = "gen_ai.agent.description"- GEN_AI_WORKFLOW_TYPE = "gen_ai.workflow.type"-```--### Business Intelligence Attributes--**Always capture OpenLIT's competitive advantages:**--```python-# Token usage and cost tracking-span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)-span.set_attribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)-span.set_attribute(SemanticConvention.GEN_AI_CLIENT_OPERATION_DURATION, duration)--# Content capture with MIME types-span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, content)-span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, response)-```--## Quality Checklist--### Phase Completion Checklist--**Phase 1 - Competitive Analysis:**-- [ ] Competitor repositories cloned and analyzed-- [ ] Integration patterns documented-- [ ] Competitive gaps identified-- [ ] OpenLIT advantages planned--**Phase 2 - Framework Analysis:**-- [ ] Built-in capabilities explored-- [ ] Method discovery completed-- [ ] Integration pattern chosen-- [ ] Operation mapping defined--**Phase 3 - Implementation Strategy:**-- [ ] Span naming convention defined-- [ ] Semantic conventions planned-- [ ] Test infrastructure designed--**Phase 4 - Implementation:**-- [ ] Test script created first-- [ ] Integration pattern implemented-- [ ] Semantic conventions used extensively-- [ ] Business intelligence captured--**Phase 5 - Testing:**-- [ ] Span hierarchy validated-- [ ] Competitive comparison passed-- [ ] Mock testing completed-- [ ] Error resilience tested--**Phase 6 - Optimization:**-- [ ] Performance overhead measured (<10%)-- [ ] Business intelligence validated-- [ ] Competitive advantages confirmed--**Phase 7 - Cleanup:**-- [ ] Code quality checks passed-- [ ] Syntax errors resolved-- [ ] Line length compliant-- [ ] Final validation completed--### Success Criteria--**Instrumentation is complete when:**-1. 🚨 **CRITICAL: Proper Span Hierarchy**: NO "all root spans" - proper parent-child relationships maintained-2. 🚨 **CRITICAL: Threading Context Fixes**: Handles ThreadPoolExecutor and async context propagation-3. ✅ **Perfect Pylint Score**: Achieves exactly 10.00/10 (not 9.5, must be perfect)-4. ✅ **Mandatory test.py**: Uses exactly "test.py" filename with venv testing-5. ✅ **Competitive Analysis**: Studied OpenInference, OpenLLMetry, AgentOps, LangFuse-6. ✅ **Target SDK Study**: Cloned and analyzed framework's internal structure-7. ✅ **Semantic Conventions**: Uses SemanticConvention extensively (no hardcoded strings)-8. ✅ **Superior Span Coverage**: Generates more/better spans than competitors-9. ✅ **Naming Convention**: Follows `{operation_type} {operation_name}` format-10. ✅ **4-File Structure**: Uses recommended Pattern 4 when needed for complex frameworks-11. ✅ **Context Caching**: Implements performance optimization patterns with __slots__-12. ✅ **Business Intelligence**: Captures comprehensive cost/token/performance data-13. ✅ **Performance**: Overhead <10% with benchmarking vs competitors-14. ✅ **Error Resilience**: Handles framework version differences gracefully--**⚠️ PRIORITY ORDER**: Span hierarchy (#1) and threading fixes (#2) are MOST CRITICAL - everything else is secondary.--**OpenLIT Competitive Advantages Delivered:**-- 🎯 **Superior Business Intelligence**: Detailed cost tracking, token usage, performance metrics-- 🔗 **Cross-System Integration**: Links to LLM provider spans with proper hierarchy-- 🎨 **Enhanced Observability**: MIME types, structured content capture-- 🛡️ **Error Resilience**: Graceful degradation across framework versions-- 📊 **Comprehensive Coverage**: More spans and attributes than all competitors-- ⚡ **Performance Leadership**: Context caching and optimization patterns-- 🏗️ **Flexible Architecture**: Adapts to different framework patterns (processor, callback, wrapper)\ No newline at end of filediff --git a/contributors/operator-ui.md b/contributors/operator-ui.mddeleted file mode 100644index 798fa570..00000000--- a/contributors/operator-ui.md+++ /dev/null@@ -1,1330 +0,0 @@-# OpenLIT Operator External Management System--> **⚠️ IMPORTANT**: This is an **optional feature** that does not affect existing operator functionality. Users can continue using the operator exactly as before without any changes.--## 📋 **Table of Contents**-1. [Overview](#overview)-2. [Backward Compatibility](#backward-compatibility)-3. [Architecture](#architecture)-4. [System Components](#system-components)-5. [Data Flow](#data-flow)-6. [Implementation Details](#implementation-details)-7. [Configuration](#configuration)-8. [API Specifications](#api-specifications)-9. [Deployment Guide](#deployment-guide)-10. [User Workflows](#user-workflows)-11. [Security Considerations](#security-considerations)-12. [Multi-Operator Scenarios](#multi-operator-scenarios)-13. [Troubleshooting Guide](#troubleshooting-guide)-------## 🎯 **Overview**--The External Management System is an **optional add-on** that enables remote management of OpenLIT operators through a web interface. This system allows platform teams to manage multiple customer clusters from a centralized UI while maintaining complete security isolation.--### **Key Benefits**-- **🌐 Centralized Management**: Single UI for multiple customer clusters-- **🔒 Security First**: No inbound access required to customer clusters-- **⚡ Real-time Monitoring**: Live workload discovery and instrumentation status-- **🎯 Zero Impact**: Completely optional - existing workflows unchanged-- **📈 Scalable**: Supports multiple operators per cluster and multiple clusters per UI-------## ✅ **Backward Compatibility**--### **Non-Breaking Design Principles**--**✅ Completely Optional**-```yaml-# Default behavior - external UI disabled-sync:- enabled: false # Default: false-```--**✅ Graceful Degradation**-```go-// Operator continues normal operation even if external UI is unreachable-if config.ExternalUI.Enabled {- go externalUISync.Start(ctx) // Runs in background-}-// Main operator functionality unaffected-```--**✅ No Changes to Existing APIs**-- All existing operator functionality remains identical-- AutoInstrumentation CRs work exactly the same-- Webhook behavior unchanged-- RBAC permissions unchanged (except optional additions)--**✅ Zero Configuration Impact**-```yaml-# Existing users need NO configuration changes-# Their operator deployments continue working as-is-```--### **Migration Path**-1. **Phase 1**: Deploy enhanced operator (external UI disabled by default)-2. **Phase 2**: Optionally enable external UI for specific clusters-3. **Phase 3**: Gradually migrate management workflows to UI (optional)-------## 🏗️ **Architecture**--### **High-Level Architecture**-```-┌─────────────────────────────────────┐ ┌─────────────────────────────────────┐-│ Customer K8s │ │ External UI Service │-│ (Private Network) │ │ (Internet Accessible) │-│ │ │ │-│ ┌───────────────────────────────┐ │ │ ┌───────────────────────────────┐ │-│ │ OpenLIT Operator │ │ │ │ Web Interface │ │-│ │ (Enhanced, Optional) │ │ │ │ (HTML/CSS/JS) │ │-│ │ │ │ │ │ │ │-│ │ ┌─────────────────────────┐ │ │ │ └───────────────────────────────┘ │-│ │ │ Workload Discovery │ │ │ │ │-│ │ │ (Unchanged) │ │ │ │ ┌───────────────────────────────┐ │-│ │ └─────────────────────────┘ │ │ │ │ REST API Server │ │-│ │ │ │ │ │ - Receive workload data │ │-│ │ ┌─────────────────────────┐ │ │ │ │ - Queue commands │ │-│ │ │ AutoInstrumentation │ │ │ │ │ - Serve UI responses │ │-│ │ │ Management (Unchanged) │ │ │ │ └───────────────────────────────┘ │-│ │ └─────────────────────────┘ │ │ │ │-│ │ │ │ │ ┌───────────────────────────────┐ │-│ │ ┌─────────────────────────┐ │ │ │ │ Data Storage │ │-│ │ │ External UI Sync │ │──┼────┼─►│ - Cluster states │ │-│ │ │ (NEW - Optional) │ │ │ │ │ - Workload inventory │ │-│ │ └─────────────────────────┘ │ │ │ │ - Command queue │ │-│ └───────────────────────────────┘ │ │ └───────────────────────────────┘ │-└─────────────────────────────────────┘ └─────────────────────────────────────┘- (No Inbound Access) (Internet Accessible)-```--### **Communication Flow**-- **Outbound Only**: Operator initiates all connections to external UI-- **Push Model**: Operator pushes workload discovery data every 30 seconds (when enabled)-- **Pull Model**: Operator pulls pending commands from external UI (when enabled)-- **Stateless**: External UI maintains cluster state; operator executes commands-------## 🧩 **System Components**--### **1. Enhanced OpenLIT Operator (Backward Compatible)**-**Location**: Customer Kubernetes cluster-**Changes**: Additive only - new optional modules--**Core Functionality (Unchanged)**:-- Zero-code auto-instrumentation-- AutoInstrumentation CR management-- Webhook-based pod injection-- Multi-provider support--**New Optional Modules**:-```-operator/-├── internal/-│ ├── external/ # NEW - Optional external UI sync-│ │ ├── sync.go-│ │ ├── client.go-│ │ └── types.go-│ ├── discovery/ # NEW - Optional workload discovery-│ │ ├── workloads.go-│ │ ├── instrumentation.go-│ │ └── status.go-│ └── commands/ # NEW - Optional command execution-│ ├── executor.go-│ └── handlers.go-```--**Integration Pattern**:-```go-// main.go - Non-breaking integration-func main() {- // Existing operator initialization (unchanged)- mgr := setupManager()- setupWebhooks(mgr)- setupControllers(mgr)-- // NEW - Optional external UI integration- if config.ExternalUI.Enabled {- externalUISync := external.NewExternalUISync(config.ExternalUI, mgr.GetClient())- go externalUISync.Start(ctx) // Non-blocking- log.Info("External UI sync enabled")- }-- // Existing operator start (unchanged)- mgr.Start(ctx)-}-```--### **2. External UI Service**-**Location**: Internet-accessible environment-**Purpose**: Centralized management interface--**Components**:-```-external-ui-service/-├── main.go # HTTP server and routing-├── api/-│ ├── operators.go # APIs for operator communication-│ ├── ui.go # APIs for web interface-│ ├── middleware.go # Authentication and logging-│ └── types.go # Shared data structures-├── storage/-│ ├── memory.go # In-memory storage implementation-│ ├── interface.go # Storage abstraction-│ └── models.go # Data models-├── web/-│ ├── static/-│ │ ├── index.html # Main dashboard-│ │ ├── styles.css # UI styling-│ │ └── app.js # Frontend JavaScript-│ └── templates/-│ └── cluster.html # Cluster detail views-└── config/- └── config.go # Service configuration-```-------## 🔄 **Data Flow**--### **1. Discovery and Sync (Operator → External UI)**-**Frequency**: Every 30 seconds (when enabled)-**Direction**: Operator pushes to External UI--```mermaid-sequenceDiagram- participant O as Operator- participant K as Kubernetes API- participant E as External UI-- Note over O: Only when sync.enabled = true-- loop Every 30 seconds- O->>K: List Deployments/StatefulSets/DaemonSets- K->>O: Return workloads- O->>K: List AutoInstrumentation CRs- K->>O: Return configurations- O->>O: Analyze instrumentation status- O->>E: POST /api/operators/sync (workload data)- E->>O: 200 OK- end-```--**Sync Payload Structure**:-```json-{- "timestamp": "2024-01-15T10:30:00Z",- "operator_namespaces": ["production", "staging"],- "workloads": [- {- "name": "my-fastapi-app",- "namespace": "production",- "type": "Deployment",- "replicas": "3/3",- "status": "Running",- "labels": {- "app": "my-fastapi-app",- "team": "backend",- "framework": "fastapi",- "environment": "production"- },- "instrumentation": {- "status": "not_instrumented",- "matching_configs": [],- "requires_restart": false- },- "created_at": "2024-01-10T08:15:00Z",- "image": "my-registry/fastapi-app:v1.2.3",- "ports": [8000]- }- ],- "auto_instrumentations": [- {- "name": "backend-team-instrumentation",- "namespace": "default",- "selector": {- "matchLabels": {- "team": "backend",- "environment": "production"- }- },- "provider": "openlit",- "status": "active"- }- ]-}-```--### **2. Command Management (External UI → Operator)**-**Frequency**: Every 30 seconds (operator polls when enabled)-**Direction**: Operator pulls from External UI--```mermaid-sequenceDiagram- participant U as User- participant E as External UI- participant O as Operator- participant K as Kubernetes API-- U->>E: Click "Instrument my-fastapi-app"- E->>E: Queue instrumentation command-- Note over O: Only when sync.enabled = true-- loop Every 30 seconds- O->>E: GET /api/operators/commands- E->>O: Return pending commands- O->>O: Filter commands for my namespaces- O->>K: Create AutoInstrumentation CR- K->>O: CR created successfully- O->>E: DELETE /api/operators/commands/{id} (ack)- E->>O: Command acknowledged- end-```--**Command Structure**:-```json-{- "id": "cmd_123456789",- "type": "create_instrumentation",- "timestamp": "2024-01-15T10:35:00Z",- "namespace": "production",- "workload": "my-fastapi-app",- "data": {- "auto_instrumentation_name": "my-fastapi-app-instrumentation",- "selector": {- "matchLabels": {- "app": "my-fastapi-app"- }- },- "provider": "openlit",- "otlp_endpoint": "http://openlit.openlit.svc.cluster.local:4318"- }-}-```-------## 🛠️ **Implementation Details**--### **Operator External Sync Module (Non-Breaking)**--```go-// internal/external/sync.go-type ExternalUISync struct {- kubeclient kubernetes.Interface- autoInstrClient v1alpha1client.AutoInstrumentationInterface- externalURL string- apiKey string- targetNamespaces []string- syncInterval time.Duration- httpClient *http.Client- logger logr.Logger- enabled bool // Key: can be disabled-}--func NewExternalUISync(config ExternalUIConfig, kubeclient kubernetes.Interface) *ExternalUISync {- return &ExternalUISync{- enabled: config.Enabled, // Respects configuration- kubeclient: kubeclient,- externalURL: config.Endpoint,- apiKey: config.APIKey,- targetNamespaces: config.TargetNamespaces,- syncInterval: config.SyncInterval,- httpClient: &http.Client{- Timeout: 30 * time.Second,- Transport: &http.Transport{- MaxIdleConns: 10,- IdleConnTimeout: 90 * time.Second,- DisableCompression: false,- },- },- logger: ctrl.Log.WithName("external-ui-sync"),- }-}--func (s *ExternalUISync) Start(ctx context.Context) error {- if !s.enabled {- s.logger.Info("External UI sync disabled, skipping")- return nil // Graceful no-op- }-- s.logger.Info("Starting external UI sync", "endpoint", s.externalURL, "interval", s.syncInterval)-- ticker := time.NewTicker(s.syncInterval)- defer ticker.Stop()-- // Initial sync with error handling- if err := s.performSync(ctx); err != nil {- s.logger.Error(err, "Initial sync failed, continuing anyway")- // Note: Operator continues running even if external UI fails- }-- for {- select {- case <-ctx.Done():- return ctx.Err()- case <-ticker.C:- if err := s.performSync(ctx); err != nil {- s.logger.Error(err, "Sync cycle failed, will retry")- // Continue running - external UI issues don't break operator- }- }- }-}--func (s *ExternalUISync) performSync(ctx context.Context) error {- // Graceful error handling - don't break operator if external UI fails- defer func() {- if r := recover(); r != nil {- s.logger.Error(fmt.Errorf("sync panic: %v", r), "External UI sync panicked, continuing")- }- }()-- // 1. Discover current workload state- state, err := s.discoverClusterState(ctx)- if err != nil {- return fmt.Errorf("discovery failed: %w", err)- }-- // 2. Push state to external UI (with timeout)- if err := s.pushClusterState(ctx, state); err != nil {- return fmt.Errorf("push failed: %w", err)- }-- // 3. Pull and execute commands (with timeout)- if err := s.pullAndExecuteCommands(ctx); err != nil {- return fmt.Errorf("command execution failed: %w", err)- }-- return nil-}--// Namespace filtering - only act on assigned namespaces-func (s *ExternalUISync) isMyNamespace(namespace string) bool {- for _, ns := range s.targetNamespaces {- if ns == namespace {- return true- }- }- return false-}-```--### **Workload Discovery Engine**--```go-// internal/discovery/workloads.go-type WorkloadDiscovery struct {- kubeclient kubernetes.Interface- autoInstrClient v1alpha1client.AutoInstrumentationInterface- targetNamespaces []string- logger logr.Logger-}--func (d *WorkloadDiscovery) DiscoverAll(ctx context.Context) (*ClusterState, error) {- state := &ClusterState{- Timestamp: time.Now(),- OperatorNamespaces: d.targetNamespaces,- Workloads: []WorkloadInfo{},- AutoInstrumentations: []AutoInstrumentationInfo{},- }-- // Discover workloads in each target namespace- for _, namespace := range d.targetNamespaces {- workloads, err := d.discoverNamespace(ctx, namespace)- if err != nil {- d.logger.Error(err, "Failed to discover namespace", "namespace", namespace)- continue // Continue with other namespaces- }- state.Workloads = append(state.Workloads, workloads...)- }-- // Get AutoInstrumentation configurations- autoInstrumentations, err := d.getAutoInstrumentations(ctx)- if err != nil {- d.logger.Error(err, "Failed to get AutoInstrumentation CRs")- } else {- state.AutoInstrumentations = autoInstrumentations- }-- // Analyze instrumentation status for each workload- d.analyzeInstrumentationStatus(state)-- return state, nil-}--func (d *WorkloadDiscovery) discoverNamespace(ctx context.Context, namespace string) ([]WorkloadInfo, error) {- var allWorkloads []WorkloadInfo-- // Get Deployments- deployments, err := d.kubeclient.AppsV1().Deployments(namespace).List(ctx, metav1.ListOptions{})- if err != nil {- return nil, err- }-- for _, deploy := range deployments.Items {- workload := d.deploymentToWorkloadInfo(&deploy)- allWorkloads = append(allWorkloads, workload)- }-- // Get StatefulSets- statefulsets, err := d.kubeclient.AppsV1().StatefulSets(namespace).List(ctx, metav1.ListOptions{})- if err != nil {- return nil, err- }-- for _, sts := range statefulsets.Items {- workload := d.statefulSetToWorkloadInfo(&sts)- allWorkloads = append(allWorkloads, workload)- }-- // Get DaemonSets- daemonsets, err := d.kubeclient.AppsV1().DaemonSets(namespace).List(ctx, metav1.ListOptions{})- if err != nil {- return nil, err- }-- for _, ds := range daemonsets.Items {- workload := d.daemonSetToWorkloadInfo(&ds)- allWorkloads = append(allWorkloads, workload)- }-- return allWorkloads, nil-}-```--### **External UI Service Implementation**--```go-// main.go-func main() {- storage := storage.NewMemoryStorage()-- r := gin.Default()-- // CORS middleware- r.Use(func(c *gin.Context) {- c.Header("Access-Control-Allow-Origin", "*")- c.Header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE, OPTIONS")- c.Header("Access-Control-Allow-Headers", "Content-Type, Authorization")-- if c.Request.Method == "OPTIONS" {- c.AbortWithStatus(204)- return- }-- c.Next()- })-- // Serve static files- r.Static("/static", "./web/static")- r.LoadHTMLGlob("web/templates/*")-- // Main UI route- r.GET("/", func(c *gin.Context) {- c.HTML(200, "index.html", gin.H{- "title": "OpenLIT Operator Management",- })- })-- // API routes for operators- operatorAPI := r.Group("/api/operators")- operatorAPI.Use(AuthMiddleware())- {- operatorAPI.POST("/sync", handleOperatorSync(storage))- operatorAPI.GET("/commands", handleGetCommands(storage))- operatorAPI.DELETE("/commands/:id", handleAckCommand(storage))- }-- // API routes for UI- uiAPI := r.Group("/api/ui")- {- uiAPI.GET("/clusters", handleGetClusters(storage))- uiAPI.GET("/clusters/:apikey/workloads", handleGetWorkloads(storage))- uiAPI.POST("/clusters/:apikey/instrument", handleCreateInstrumentation(storage))- uiAPI.POST("/clusters/:apikey/restart", handleRestartWorkload(storage))- }-- port := os.Getenv("PORT")- if port == "" {- port = "8080"- }-- log.Printf("Starting server on port %s", port)- r.Run(":" + port)-}--// api/operators.go-func handleOperatorSync(storage Storage) gin.HandlerFunc {- return func(c *gin.Context) {- apiKey := extractAPIKey(c)- if apiKey == "" {- c.JSON(401, gin.H{"error": "Invalid or missing API key"})- return- }-- var syncData ClusterSyncPayload- if err := c.ShouldBindJSON(&syncData); err != nil {- c.JSON(400, gin.H{"error": "Invalid JSON payload", "details": err.Error()})- return- }-- // Update cluster state- clusterInfo := &ClusterInfo{- APIKey: apiKey,- LastSeen: time.Now(),- OperatorNamespaces: syncData.OperatorNamespaces,- Workloads: syncData.Workloads,- AutoInstrumentations: syncData.AutoInstrumentations,- }-- if err := storage.UpdateCluster(apiKey, clusterInfo); err != nil {- c.JSON(500, gin.H{"error": "Failed to update cluster state"})- return- }-- c.JSON(200, gin.H{- "status": "success",- "timestamp": time.Now(),- "workloads_received": len(syncData.Workloads),- })- }-}--func handleGetCommands(storage Storage) gin.HandlerFunc {- return func(c *gin.Context) {- apiKey := extractAPIKey(c)- if apiKey == "" {- c.JSON(401, gin.H{"error": "Invalid or missing API key"})- return- }-- commands, err := storage.GetPendingCommands(apiKey)- if err != nil {- c.JSON(500, gin.H{"error": "Failed to get commands"})- return- }-- c.JSON(200, gin.H{- "commands": commands,- "count": len(commands),- })- }-}-```-------## ⚙️ **Configuration**--### **Operator Configuration (Backward Compatible)**--```yaml-# operator-config.yaml-apiVersion: v1-kind: ConfigMap-metadata:- name: openlit-operator-config- namespace: openlit-data:- config.yaml: |- # Core operator settings (UNCHANGED)- operator:- # Namespaces this operator instance monitors- target_namespaces: ["production", "staging"]-- # AutoInstrumentation settings (UNCHANGED)- auto_instrumentation:- default_provider: "openlit"- default_otlp_endpoint: "http://openlit.openlit.svc.cluster.local:4318"-- # NEW - External UI integration (OPTIONAL)- sync:- enabled: false # DEFAULT: disabled for backward compatibility- endpoint: "https://openlit-management.company.com"- sync_interval: "30s"-- # Retry and timeout settings- timeout: "30s"- max_retries: 3- retry_delay: "5s"-- # HTTP client settings- tls_skip_verify: false- max_idle_connections: 10-```--### **Enabling External UI (Optional)**--```yaml-# To enable external UI, users must explicitly opt-in:-sync:- enabled: true # Explicitly enable- endpoint: "https://openlit-management.company.com"- # API key provided via secret-```--```yaml-# external-ui-secret.yaml (ONLY needed if external UI enabled)-apiVersion: v1-kind: Secret-metadata:- name: external-ui-secret- namespace: openlit-type: Opaque-data:- api-key: b2xpdF9ha18xMjM0NTY3ODkwYWJjZGVm # base64 encoded: olit_ak_1234567890abcdef-```--### **External UI Service Configuration**--```yaml-# external-ui-config.yaml-server:- port: 8080- read_timeout: "30s"- write_timeout: "30s"- idle_timeout: "60s"--storage:- type: "memory" # Options: memory, redis, postgres-- # For Redis storage- redis:- host: "redis:6379"- password: ""- db: 0-- # For PostgreSQL storage- postgres:- host: "postgres"- port: 5432- database: "openlit_ui"- username: "postgres"- password: "secretpass"--clusters:- max_age: "5m" # Mark cluster as stale after 5 minutes- cleanup_interval: "1h" # Clean up stale data every hour--security:- # API keys for cluster authentication- api_keys:- - key: "olit_ak_1234567890abcdef"- name: "Acme Corp - Production"- customer: "Acme Corp"- environment: "production"- - key: "olit_ak_fedcba0987654321"- name: "Acme Corp - Staging"- customer: "Acme Corp"- environment: "staging"--logging:- level: "info" # debug, info, warn, error- format: "json" # json, text-```-------## 📡 **API Specifications**--### **Operator APIs (Called by Operator)**--#### **POST /api/operators/sync**-Push workload data to external UI.--**Headers:**-- `Authorization: Bearer {api_key}`-- `Content-Type: application/json`--**Request Body:**-```json-{- "timestamp": "2024-01-15T10:30:00Z",- "operator_namespaces": ["production", "staging"],- "workloads": [/* WorkloadInfo objects */],- "auto_instrumentations": [/* AutoInstrumentationInfo objects */]-}-```--**Response:**-```json-{- "status": "success",- "timestamp": "2024-01-15T10:30:01Z",- "workloads_received": 5-}-```--#### **GET /api/operators/commands**-Get pending commands for this cluster.--**Headers:**-- `Authorization: Bearer {api_key}`--**Response:**-```json-{- "commands": [- {- "id": "cmd_123456789",- "type": "create_instrumentation",- "timestamp": "2024-01-15T10:35:00Z",- "namespace": "production",- "workload": "my-fastapi-app",- "data": {/* command-specific data */}- }- ],- "count": 1-}-```--#### **DELETE /api/operators/commands/{id}**-Acknowledge command execution.--**Headers:**-- `Authorization: Bearer {api_key}`--**Response:**-```json-{- "status": "acknowledged",- "command_id": "cmd_123456789"-}-```--### **UI APIs (Called by Web Interface)**--#### **GET /api/ui/clusters**-Get all connected clusters.--**Response:**-```json-[- {- "name": "Acme Corp - Production",- "customer": "Acme Corp",- "environment": "production",- "last_seen": "2024-01-15T10:30:00Z",- "namespaces": ["production", "staging"],- "workload_count": 8,- "instrumented_count": 3,- "status": "online"- }-]-```--#### **GET /api/ui/clusters/{apikey}/workloads**-Get workloads for specific cluster.--**Response:**-```json-{- "cluster": {- "name": "Acme Corp - Production",- "last_seen": "2024-01-15T10:30:00Z"- },- "workloads": [/* WorkloadInfo objects */]-}-```--#### **POST /api/ui/clusters/{apikey}/instrument**-Create instrumentation command.--**Request Body:**-```json-{- "workload": "my-fastapi-app",- "namespace": "production",- "selector": {- "matchLabels": {- "app": "my-fastapi-app"- }- },- "auto_instrumentation_name": "my-fastapi-app-instrumentation",- "provider": "openlit"-}-```--**Response:**-```json-{- "status": "queued",- "command_id": "cmd_123456789",- "estimated_execution": "30s"-}-```-------## 🚀 **Deployment Guide**--### **1. Deploy External UI Service (Optional)**--```bash-# Build the service-git clone https://github.com/openlit/external-ui-service-cd external-ui-service-docker build -t openlit-external-ui:latest .--# Run with Docker-docker run -d \- --name openlit-external-ui \- -p 8080:8080 \- -v $(pwd)/config.yaml:/app/config.yaml \- openlit-external-ui:latest--# Or deploy to Kubernetes-kubectl apply -f deployments/external-ui-deployment.yaml-```--**External UI Kubernetes Deployment:**-```yaml-apiVersion: apps/v1-kind: Deployment-metadata:- name: openlit-external-ui- namespace: openlit-management-spec:- replicas: 2- selector:- matchLabels:- app: openlit-external-ui- template:- metadata:- labels:- app: openlit-external-ui- spec:- containers:- - name: external-ui- image: openlit-external-ui:latest- ports:- - containerPort: 8080- env:- - name: CONFIG_PATH- value: "/app/config.yaml"- volumeMounts:- - name: config- mountPath: /app/config.yaml- subPath: config.yaml- resources:- requests:- memory: "64Mi"- cpu: "50m"- limits:- memory: "128Mi"- cpu: "100m"- volumes:- - name: config- configMap:- name: external-ui-config------apiVersion: v1-kind: Service-metadata:- name: openlit-external-ui-service- namespace: openlit-management-spec:- selector:- app: openlit-external-ui- ports:- - port: 80- targetPort: 8080- type: LoadBalancer------apiVersion: networking.k8s.io/v1-kind: Ingress-metadata:- name: openlit-external-ui-ingress- namespace: openlit-management- annotations:- kubernetes.io/ingress.class: nginx- cert-manager.io/cluster-issuer: letsencrypt-prod-spec:- tls:- - hosts:- - openlit-management.company.com- secretName: openlit-external-ui-tls- rules:- - host: openlit-management.company.com- http:- paths:- - path: /- pathType: Prefix- backend:- service:- name: openlit-external-ui-service- port:- number: 80-```--### **2. Deploy Enhanced Operator (Backward Compatible)**--```bash-# Existing users: NO CHANGES NEEDED-# Operator works exactly as before--# For external UI (optional):-# Create API key secret-kubectl create secret generic external-ui-secret \- --from-literal=api-key=olit_ak_1234567890abcdef \- -n openlit--# Update configuration to enable external UI-kubectl apply -f operator-config-with-external-ui.yaml--# Deploy enhanced operator (backward compatible)-kubectl apply -f enhanced-operator-deployment.yaml-```--**Enhanced Operator Deployment (Backward Compatible):**-```yaml-apiVersion: apps/v1-kind: Deployment-metadata:- name: openlit-operator- namespace: openlit-spec:- replicas: 1- selector:- matchLabels:- app: openlit-operator- template:- metadata:- labels:- app: openlit-operator- spec:- serviceAccountName: openlit-operator- containers:- - name: operator- image: openlit-operator:latest-with-external-ui # Enhanced version- command:- - /operator- env:- # External UI environment variables (with defaults for backward compatibility)- - name: EXTERNAL_UI_ENABLED- valueFrom:- configMapKeyRef:- name: openlit-operator-config- key: external_ui_enabled- optional: true # Optional for backward compatibility- - name: EXTERNAL_UI_ENDPOINT- valueFrom:- configMapKeyRef:- name: openlit-operator-config- key: external_ui_endpoint- optional: true # Optional for backward compatibility- - name: EXTERNAL_UI_API_KEY- valueFrom:- secretKeyRef:- name: external-ui-secret- key: api-key- optional: true # Optional for backward compatibility- - name: TARGET_NAMESPACES- valueFrom:- configMapKeyRef:- name: openlit-operator-config- key: target_namespaces- resources:- requests:- memory: "64Mi"- cpu: "50m"- limits:- memory: "128Mi"- cpu: "100m"- volumeMounts:- - name: config- mountPath: /etc/operator/config.yaml- subPath: config.yaml- volumes:- - name: config- configMap:- name: openlit-operator-config-```--### **3. Verify Deployment**--```bash-# Check external UI service (if deployed)-curl -I https://openlit-management.company.com-# Should return 200 OK--# Check operator logs-kubectl logs -n openlit -l app=openlit-operator -f-# Should show successful sync messages (if external UI enabled)-# Should show normal operation logs (if external UI disabled)--# Check external UI shows cluster (if enabled)-curl https://openlit-management.company.com/api/ui/clusters-# Should show your cluster in the list (if external UI enabled)-```-------## 👤 **User Workflows**--### **Existing Users (No Changes)**--1. **Continue as before**: All existing workflows work exactly the same-2. **AutoInstrumentation CRs**: Create and manage as before-3. **kubectl operations**: All commands work unchanged-4. **Monitoring**: Existing observability unchanged--### **New External UI Workflows (Optional)**--#### **Setup Workflow**--1. **Platform Admin**: Deploy external UI service (optional)-2. **Platform Admin**: Generate API key for customer cluster (optional)-3. **Customer Admin**: Enable external UI in operator config (optional)-4. **Customer Admin**: Deploy enhanced operator (backward compatible)-5. **Platform Admin**: Verify cluster appears in external UI within 30 seconds (if enabled)--#### **Daily Management Workflow**--1. **DevOps Engineer**: Opens external UI in browser-2. **View Clusters**: See all connected customer clusters and their status-3. **Select Cluster**: Click on specific cluster to view workloads-4. **Review Workloads**: See instrumentation status for each workload-5. **Take Actions**: Instrument new workloads or manage existing ones--#### **Instrumentation Workflow**--1. **Identify Target**: Find non-instrumented workload in UI-2. **Click "Instrument"**: UI shows instrumentation dialog-3. **Choose Labels**: Select which labels to use for AutoInstrumentation selector- ```- Available Labels:- ☑️ app: my-fastapi-app- ☐ team: backend- ☐ environment: production- ```-4. **Preview Impact**: UI shows which other workloads would be affected- ```- ✅ Will only instrument: my-fastapi-app- ⚠️ Would also instrument: user-service, auth-service (if team selected)- ```-5. **Create Configuration**: Generate AutoInstrumentation CR with chosen selectors-6. **Monitor Progress**: Watch operator execute command (30-60 seconds)-7. **Verify Success**: Confirm workload shows as instrumented-------## 🔄 **Multi-Operator Scenarios**--### **Scenario 1: Team-Based Operator Separation**--```-┌─────────────────────────────────────────────────────────────┐-│ Customer K8s Cluster │-│ │-│ ┌─────────────────────┐ ┌─────────────────────┐ │-│ │ Backend Team │ │ Frontend Team │ │ ┌─────────────────┐-│ │ Operator │ │ Operator │ │────►│ External UI │-│ │ │ │ │ │ │ │-│ │ Manages: │ │ Manages: │ │ │ Shows: │-│ │ - backend ns │ │ - frontend ns │ │ │ - All namespaces│-│ │ - api ns │ │ - mobile ns │ │ │ - Unified view │-│ │ - auth ns │ │ - web ns │ │ │ │-│ └─────────────────────┘ └─────────────────────┘ │ └─────────────────┘-└─────────────────────────────────────────────────────────────┘-```--**Configuration**:-```yaml-# Backend team operator-operator:- target_namespaces: ["backend", "api", "auth"]-sync:- enabled: true # Optional- api_key: "olit_ak_backend_team_12345"--# Frontend team operator-operator:- target_namespaces: ["frontend", "mobile", "web"]-sync:- enabled: true # Optional- api_key: "olit_ak_frontend_team_67890"-```--**External UI View**:-```-Customer Cluster - Production-├── backend namespace (Backend Team)-│ ├── user-service ✅ Instrumented-│ └── payment-service ❌ Not Instrumented-├── frontend namespace (Frontend Team)-│ ├── web-app ✅ Instrumented-│ └── mobile-app ❌ Not Instrumented-└── api namespace (Backend Team)- └── gateway-service ✅ Instrumented-```--### **Scenario 2: Environment-Based Separation**--```-┌─────────────────────────────────────────────────────────────┐-│ Customer K8s Cluster │-│ │-│ ┌─────────────────────┐ ┌─────────────────────┐ │-│ │ Production │ │ Staging │ │ ┌─────────────────┐-│ │ Operator │ │ Operator │ │────►│ External UI │-│ │ │ │ │ │ │ │-│ │ Manages: │ │ Manages: │ │ │ Shows: │-│ │ - prod-backend │ │ - staging-backend │ │ │ - All envs │-│ │ - prod-frontend │ │ - staging-frontend │ │ │ - Per namespace │-│ │ - prod-api │ │ - staging-api │ │ │ │-│ └─────────────────────┘ └─────────────────────┘ │ └─────────────────┘-└─────────────────────────────────────────────────────────────┘-```--### **Command Routing Logic**--```go-func (s *ExternalUISync) executeRelevantCommands(commands []Command) {- for _, cmd := range commands {- // Only execute commands for namespaces I manage- if s.isMyNamespace(cmd.Namespace) {- s.logger.Info("Executing command", "command", cmd.ID, "namespace", cmd.Namespace)-- if err := s.executeCommand(cmd); err != nil {- s.logger.Error(err, "Command execution failed", "command", cmd.ID)- continue- }-- // Acknowledge successful execution- s.acknowledgeCommand(cmd.ID)- } else {- s.logger.Debug("Ignoring command for other namespace",- "command", cmd.ID, "namespace", cmd.Namespace)- }- }-}-```--### **Conflict Resolution**--**Overlapping Namespace Management**:-- **Prevention**: Validate operator configurations to prevent namespace overlap-- **Detection**: Monitor for duplicate workload reports from multiple operators-- **Resolution**: Implement operator priority or explicit namespace ownership--**Command Ordering**:-- **Idempotent Operations**: Ensure commands can be safely executed multiple times-- **State Reconciliation**: Operators should verify current state before applying changes-- **Error Handling**: Failed commands should be retryable without side effects-------## 🔧 **Troubleshooting Guide**--### **Common Issues**--**External UI Not Working (Operator Still Works)**:-1. Check if external UI is enabled in config-2. Verify API key validity and network connectivity-3. Review operator logs for external UI sync errors-4. Confirm external UI endpoint accessibility-5. **Note**: Operator continues normal operation even if external UI fails--**Operator Not Syncing (External UI Enabled)**:-1. Check API key validity and network connectivity-2. Verify external UI endpoint accessibility-3. Review operator logs for authentication errors-4. Confirm target namespaces are accessible--**Commands Not Executing**:-1. Verify namespace ownership (operator only acts on its namespaces)-2. Check AutoInstrumentation CR creation permissions-3. Review command format and required fields-4. Monitor command acknowledgment in external UI--**UI Not Showing Clusters**:-1. Confirm API keys are properly configured-2. Check external UI storage and logs-3. Verify cluster heartbeat frequency-4. Review network connectivity from operators--### **Debugging Steps**--```bash-# Check if external UI is enabled-kubectl get configmap openlit-operator-config -n openlit -o yaml | grep external_ui--# Check operator sync status (if external UI enabled)-kubectl logs -n openlit -l app=openlit-operator | grep "external-ui-sync"--# Check operator normal operation (always works)-kubectl logs -n openlit -l app=openlit-operator | grep -E "(controller|webhook)"--# Verify API key configuration (if external UI enabled)-kubectl get secret external-ui-secret -n openlit -o yaml--# Test external UI connectivity (if external UI enabled)-curl -H "Authorization: Bearer $API_KEY" \- https://external-ui.company.com/api/operators/commands--# Check AutoInstrumentation CRs (always works)-kubectl get autoinstrumentation -A--# Monitor workload changes (always works)-kubectl get pods -A --watch-```--### **Backward Compatibility Verification**--```bash-# Verify existing AutoInstrumentation CRs still work-kubectl apply -f existing-autoinstrumentation.yaml-kubectl get autoinstrumentation--# Verify webhook still instruments pods-kubectl apply -f test-deployment.yaml-kubectl get pods -l app=test-deployment -o yaml | grep -A 10 initContainers--# Verify operator health-kubectl get pods -n openlit -l app=openlit-operator-```-------## 📝 **Migration Guide**--### **For Existing Users**--**No Action Required**:-- Existing operator deployments continue working unchanged-- All AutoInstrumentation CRs function exactly as before-- Webhook behavior remains identical-- No configuration changes needed--**Optional Migration to Enhanced Operator**:-1. **Update operator image** to enhanced version (backward compatible)-2. **Keep external UI disabled** (default behavior)-3. **Verify normal operation** continues unchanged-4. **Optionally enable external UI** later if desired--### **For New Users**--**Standard Deployment**:-- Deploy enhanced operator with external UI disabled (default)-- Use traditional kubectl/YAML workflows-- Optionally enable external UI when ready--**External UI Deployment**:-- Deploy external UI service-- Generate API keys-- Enable external UI in operator config-- Access centralized management interface-------This external management system provides a powerful optional enhancement to the OpenLIT operator while maintaining complete backward compatibility and zero impact on existing workflows. Users can adopt it at their own pace, and the operator continues to function perfectly even if the external UI is unavailable.diff --git a/contributors/otel-span-events-and-attributes.md b/contributors/otel-span-events-and-attributes.mdnew file mode 100644index 00000000..faacaead--- /dev/null+++ b/contributors/otel-span-events-and-attributes.md@@ -0,0 +1,292 @@+# OTel Span Events and Attributes: Implementation Guide++This document describes how OpenLIT implements OpenTelemetry (OTel) semantic conventions for GenAI spans and events, and how to adopt the same structure and style when adding or updating instrumentations (e.g., OpenAI → Anthropic, LiteLLM, etc.).++**Official OTel GenAI spec references:**++- [Semantic conventions for Generative AI events](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md) — event `gen_ai.client.inference.operation.details`, attribute requirement levels, well-known values.+- [Semantic conventions for generative client AI spans](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-spans.md) — inference/embeddings/retrieval spans, span name/kind, attributes, content recording.+- [GenAI docs (overview)](https://github.com/open-telemetry/semantic-conventions/tree/main/docs/gen-ai) — input/output message schemas, system instructions, tool definitions.++---++## 1. How Span Attributes Look++Spans carry OTel-aligned attributes in a consistent order. Use **constants from `openlit.semcov.SemanticConvention`** so attribute names stay in sync with the official semconv and fallbacks.++### 1.1 Common attributes (set once per span)++Set these via **`common_span_attributes()`** from `openlit.__helpers` so all instrumentations behave the same:++| Attribute (constant) | Meaning |+|-----------------------|---------|+| `GEN_AI_OPERATION` | Operation type (e.g. `chat`, `embeddings`) |+| `GEN_AI_PROVIDER_NAME` | Provider name (e.g. `openai`, `anthropic`) |+| `SERVER_ADDRESS` / `SERVER_PORT` | Server connection |+| `GEN_AI_REQUEST_MODEL` / `GEN_AI_RESPONSE_MODEL` | Request and response model |+| `GEN_AI_REQUEST_IS_STREAM` | Whether the call was streaming |+| `GEN_AI_SERVER_TBT` / `GEN_AI_SERVER_TTFT` | Time-between-token / time-to-first-token |+| `GEN_AI_SDK_VERSION` | OpenLIT/instrumentation version |+| Plus: `DEPLOYMENT_ENVIRONMENT`, `SERVICE_NAME` (from helpers) |++**Usage:** Call `common_span_attributes(scope, gen_ai_operation, GEN_AI_SYSTEM_*, server_address, server_port, request_model, response_model, environment, application_name, is_stream, tbt, ttft, version)` at the start of your common logic.++### 1.2 Request and response attributes (on span)++Set these on `scope._span` after the common attributes:++- **Request:** `GEN_AI_REQUEST_TEMPERATURE`, `GEN_AI_REQUEST_TOP_P`, `GEN_AI_REQUEST_MAX_TOKENS`, `GEN_AI_REQUEST_FREQUENCY_PENALTY`, `GEN_AI_REQUEST_PRESENCE_PENALTY`, `GEN_AI_REQUEST_STOP_SEQUENCES`, `GEN_AI_REQUEST_SEED`, `GEN_AI_REQUEST_CHOICE_COUNT` (only when ≠ 1), `GEN_AI_REQUEST_USER`, etc.+- **Response:** `GEN_AI_RESPONSE_ID`, `GEN_AI_RESPONSE_FINISH_REASON` (array), `GEN_AI_OUTPUT_TYPE` (`text` or `json`).+- **Provider-specific (TIER 2):** e.g. OpenAI: `OPENAI_REQUEST_SERVICE_TIER`, `OPENAI_RESPONSE_SERVICE_TIER`, `GEN_AI_RESPONSE_SYSTEM_FINGERPRINT`.++Use helpers (e.g. `handle_not_given()` for OpenAI) so unset request fields get consistent defaults.++### 1.3 Usage and cost attributes (on span)++Always set these when you have the data (including when 0):++- `GEN_AI_USAGE_INPUT_TOKENS`+- `GEN_AI_USAGE_OUTPUT_TOKENS`+- `GEN_AI_CLIENT_TOKEN_USAGE` (input + output)+- `GEN_AI_USAGE_COST`+- **Cached tokens (set even when 0):**+ - `GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS`+ - `GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS`++Optional extensions (e.g. reasoning) can use spec-aligned or documented custom keys (e.g. `gen_ai.usage.reasoning_tokens`).++### 1.4 Message attributes (on span)++Per OTel, input/output messages on the **span** use the same **array structure** as on events:++- **`GEN_AI_INPUT_MESSAGES`** – JSON string of an array of input messages (role + parts).+- **`GEN_AI_OUTPUT_MESSAGES`** – JSON string of an array of output messages (role + parts + finish_reason).++Do **not** set these as ad-hoc strings. Use a single helper that takes structured message lists and sets both attributes so span and event stay consistent.++**Helper (per instrumentation):**++```python+def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages) if isinstance(input_messages, list) else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages) if isinstance(output_messages, list) else output_messages,+ )+```++Input/output message structures must follow the OTel gen-ai message schemas (e.g. `gen-ai-input-messages.json` / `gen-ai-output-messages.json`). Each provider implements **`build_input_messages(...)`** and **`build_output_messages(...)`** that return these structures; the same lists are then used for both span attributes and the inference event.++### 1.5 System instructions (on span)++When present, set **`GEN_AI_SYSTEM_INSTRUCTIONS`** (e.g. JSON string of instruction content). Source can be request system messages or a response field (e.g. Responses API `instructions`).++### 1.6 Tools (on span)++When the call involves tools, set:++- `GEN_AI_TOOL_NAME` (e.g. comma-separated names)+- `GEN_AI_TOOL_CALL_ID`+- `GEN_AI_TOOL_ARGS` (e.g. comma-separated string representation)++---++## 2. How Span Events Look++We emit a **single OTel GenAI inference-details event** per successful (or handled) invocation:++- **Event name:** `gen_ai.client.inference.operation.details`+ Constant: **`SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS`**++- **Event attributes:** All inference details go in **attributes**; keep **body** empty (`""`) per spec.++### 2.1 Event attribute set++The event should carry the same semantic information as the span where applicable:++- **Operation / models:** `GEN_AI_OPERATION`, `GEN_AI_REQUEST_MODEL`, `GEN_AI_RESPONSE_MODEL`+- **Server:** `SERVER_ADDRESS`, `SERVER_PORT`+- **Messages:** `GEN_AI_INPUT_MESSAGES`, `GEN_AI_OUTPUT_MESSAGES` (same array structure as on span)+- **Tools:** `GEN_AI_TOOL_DEFINITIONS` when applicable+- **Request:** temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop_sequences, seed, choice_count+- **Response:** `GEN_AI_RESPONSE_ID`, `GEN_AI_RESPONSE_FINISH_REASON`, `GEN_AI_OUTPUT_TYPE`+- **Usage:** `GEN_AI_USAGE_INPUT_TOKENS`, `GEN_AI_USAGE_OUTPUT_TOKENS`, **`GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS`**, **`GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS`** (set even when 0)+- **Other:** `GEN_AI_SYSTEM_INSTRUCTIONS`, `ERROR_TYPE` (on error)++Use the **same attribute constants** as on the span so naming stays aligned with OTel.++### 2.2 Emitting the event++- Use a **single** helper (e.g. **`emit_inference_event()`**) that:+ - Takes `event_provider`, operation name, request/response model, input/output messages, tool definitions, server address/port, and **`**extra_attrs`** for the rest.+ - Maps **extra_attrs** keys to semantic convention attribute names (e.g. `cache_read_input_tokens` → `GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS`, `system_instructions` → `GEN_AI_SYSTEM_INSTRUCTIONS`, `error_type` → `ERROR_TYPE`).+ - Builds an OTel Event via **`otel_event(name, attributes, body="")`** from `openlit.__helpers` and calls **`event_provider.emit(event)`**.++- **Extra dict keys** (so call sites stay consistent across instrumentations):+ - `response_id`, `finish_reasons`, `output_type`, `temperature`, `max_tokens`, `top_p`, `frequency_penalty`, `presence_penalty`, `stop_sequences`, `seed`, `choice_count` / `n`+ - `input_tokens`, `output_tokens`, **`cache_read_input_tokens`**, **`cache_creation_input_tokens`** (pass even when 0)+ - `system_instructions`, `error_type`++This keeps event payloads aligned with the span and makes it easy to add new instrumentations by filling the same `extra` shape and reusing the same emitter.++---++## 3. Official Spec Attribute Reference++The following tables align with the [GenAI events](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md) and [GenAI spans](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-spans.md) docs. Use these requirement levels and value types when implementing.++### 3.1 Event: `gen_ai.client.inference.operation.details`++| OTel attribute (key) | Requirement | Value type | Notes |+|----------------------|-------------|------------|--------|+| `gen_ai.operation.name` | **Required** | string | Well-known: `chat`, `embeddings`, `text_completion`, `generate_content`, etc. |+| `gen_ai.request.model` | Conditionally required if available | string | |+| `gen_ai.response.model` | Recommended | string | |+| `server.address` | Recommended | string | |+| `server.port` | Conditionally required if `server.address` set | int | |+| `gen_ai.request.temperature` | Recommended | double | |+| `gen_ai.request.top_p` | Recommended | double | |+| `gen_ai.request.max_tokens` | Recommended | int | |+| `gen_ai.request.frequency_penalty` | Recommended | double | |+| `gen_ai.request.presence_penalty` | Recommended | double | |+| `gen_ai.request.stop_sequences` | Recommended | string[] | |+| `gen_ai.request.seed` | Conditionally required if in request | int | |+| `gen_ai.request.choice.count` | Conditionally required if in request and ≠1 | int | |+| `gen_ai.response.id` | Recommended | string | |+| `gen_ai.response.finish_reasons` | Recommended | string[] | |+| `gen_ai.output.type` | Conditionally required when applicable | string | Well-known: `text`, `json`, `image`, `speech` |+| `gen_ai.usage.input_tokens` | Recommended | int | Should include cached tokens |+| `gen_ai.usage.output_tokens` | Recommended | int | |+| `gen_ai.usage.cache_creation.input_tokens` | Recommended | int | Set even when 0 |+| `gen_ai.usage.cache_read.input_tokens` | Recommended | int | Set even when 0 |+| `gen_ai.input.messages` | Opt-in | any | Structured on events; follow input-messages schema |+| `gen_ai.output.messages` | Opt-in | any | Structured on events; follow output-messages schema |+| `gen_ai.system_instructions` | Opt-in | any | Follow system-instructions schema |+| `gen_ai.tool.definitions` | Opt-in | any | Array of tool definitions |+| `gen_ai.conversation.id` | Conditionally required when available | string | |+| `error.type` | Conditionally required on error | string | Well-known: `_OTHER`; use exception type or provider error code |++### 3.2 Inference span (same attributes as above)++- **Span name:** `{gen_ai.operation.name} {gen_ai.request.model}` (e.g. `chat gpt-4o`).+- **Span kind:** `CLIENT` (or `INTERNAL` for in-process).+- **Span status:** Follow [Recording errors](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-spans.md).+- **Messages on span:** Per spec, `gen_ai.input.messages` / `gen_ai.output.messages` on spans MAY be a JSON string if structured format is not supported; on events they MUST be structured. OpenLIT uses JSON string on spans for compatibility.+- **Sampling:** Provide `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `server.address`, `server.port` at span creation time when possible.++### 3.3 Embeddings span++- `gen_ai.operation.name` = `embeddings`; span name = `embeddings {gen_ai.request.model}`.+- Attributes: `gen_ai.embeddings.dimension.count` (Recommended), `gen_ai.request.encoding_formats`, `gen_ai.usage.input_tokens`, plus common server/model/error attributes.++### 3.4 Well-known values (use these when applicable)++- **`error.type`:** `_OTHER` as fallback when no custom value.+- **`gen_ai.operation.name`:** `chat`, `embeddings`, `text_completion`, `generate_content`, `execute_tool`, `invoke_agent`, `create_agent`, `retrieval`.+- **`gen_ai.output.type`:** `text`, `json`, `image`, `speech`.+- **`gen_ai.provider.name`:** `openai`, `anthropic`, `aws.bedrock`, `gcp.gemini`, `azure.ai.openai`, etc. (see spans doc for full list).++---++## 4. Code and File Structure Reference++Keep **function names**, **argument order**, **comment headings**, and **sequence of operations** consistent across instrumentations so one pattern can be copied and maintained. The **reference implementation** is the OpenAI instrumentation: `sdk/python/src/openlit/instrumentation/openai/` (`utils.py`, `openai.py`, `async_openai.py`). New or updated instrumentations should mirror its structure, signatures, and comment style.++### 4.1 File layout (per provider)++- **`instrumentation/<provider>/`**+ - **`__init__.py`** – Instrumentor; wraps SDK entrypoints and passes `event_provider`, `capture_message_content`, etc., into the handler factory.+ - **`<provider>.py`** (e.g. `openai.py`) – Sync entrypoints that create span, extract request context, call into utils, and invoke **common_*_logic** with a **scope** object.+ - **`async_<provider>.py`** – Async counterparts.+ - **`utils.py`** – Shared helpers:+ - Provider-specific: `format_content`, `build_input_messages`, `build_output_messages`, `handle_not_given` (or equivalent), and any response/usage parsers.+ - Shared pattern: **`_set_span_messages_as_array(span, input_messages, output_messages)`**.+ - **`emit_inference_event(event_provider, operation_name, request_model, response_model, input_messages, output_messages, tool_definitions, server_address, server_port, **extra_attrs)`** – builds and emits the inference event with the key→OTel mapping above.++### 4.2 Function names and signatures (use exactly these names and argument order)++**In `openlit.__helpers` (reuse, do not redefine):** `common_span_attributes(scope, gen_ai_operation, GEN_AI_PROVIDER_NAME, server_address, server_port, request_model, response_model, environment, application_name, is_stream, tbt, ttft, version)` — docstring: "Set common span attributes for both chat and RAG operations."++**In each provider's `utils.py`:** `format_content(messages)`; `build_input_messages(messages)` (docstring: "Convert <Provider> request messages to OTel input message structure. Follows gen-ai-input-messages schema."); `build_output_messages(response_text, finish_reason, tool_calls=None)` (docstring: "Convert <Provider> response to OTel output message structure. Follows gen-ai-output-messages schema."); `_set_span_messages_as_array(span, input_messages, output_messages)` (docstring: "Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."); `emit_inference_event(event_provider, operation_name, request_model, response_model, input_messages=None, output_messages=None, tool_definitions=None, server_address=None, server_port=None, **extra_attrs)` (docstring: "Emit gen_ai.client.inference.operation.details event.").++**Common logic names:** `common_chat_logic`, `common_response_logic`, `common_embedding_logic`. Signature: `(scope, pricing_info, environment, application_name, metrics, capture_message_content, disable_metrics, version, is_stream, event_provider=None)`. Docstring: "Process <op> request and generate Telemetry."++### 4.3 Scope attributes (naming: leading underscore)++Use a **scope** object (e.g. a simple type or namespace) to carry everything from “raw response + request” to “span + event”:++- **Request/context:** `_kwargs`, `_start_time`, `_end_time`, `_server_address`, `_server_port`, etc.+- **Parsed response:** `_response_id`, `_response_model`, `_llmresponse`, `_finish_reason`, `_tools` / `_response_tools`, `_operation_type` (if you have multiple APIs).+- **Usage:** `_input_tokens`, `_output_tokens`, **`_cache_read_input_tokens`**, **`_cache_creation_input_tokens`** (set in response parser even when 0).+- **Optional:** `_reasoning_tokens`, `_instructions`, `_system_fingerprint`, `_service_tier`, etc.+- **Span:** `_span` (set by the entrypoint).++Naming: **leading underscore** for scope attributes so it’s clear they are internal to the instrumentation flow.++### 4.4 Sequence of operations inside common_*_logic (exact order and comment style)++1. **Compute cost** — `cost = get_chat_model_cost(...)`.+2. **Common Span Attributes** — `common_span_attributes(scope, ...)`.+3. **Provider-specific API type** (if any) — e.g. comment: `# OpenAI-specific API type attribute`.+4. **Span Attributes for Request parameters** — temperature, top_p, max_tokens, frequency_penalty, presence_penalty, stop_sequences, seed, user, choice_count (only when ≠1).+5. **Span Attributes for Response parameters** — response_id, finish_reason (as array), output_type.+6. **Span Attributes for Tools** (if tools) — GEN_AI_TOOL_NAME, GEN_AI_TOOL_CALL_ID, GEN_AI_TOOL_ARGS.+7. **Span Attributes for Cost and Tokens** — GEN_AI_USAGE_INPUT_TOKENS, GEN_AI_USAGE_OUTPUT_TOKENS, GEN_AI_CLIENT_TOKEN_USAGE, GEN_AI_USAGE_COST.+8. **Reasoning tokens** (optional).+9. **OTel cached token attributes (set even when 0)** — GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS, GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS when scope has them.+10. **Span Attributes for Content** (only if `capture_message_content`) — build_input_messages, build_output_messages, _set_span_messages_as_array, GEN_AI_SYSTEM_INSTRUCTIONS; then **Emit inference event** with extra containing cache_read_input_tokens, cache_creation_input_tokens (even when 0), plus response_id, finish_reasons, output_type, temperature, max_tokens, top_p, input_tokens, output_tokens, system_instructions.+11. **Span status and metrics** — `scope._span.set_status(Status(StatusCode.OK))`, then `record_completion_metrics(...)` if not disable_metrics.++### 4.5 Response parsing (before calling common_*_logic)++Use comment: `# Handle token usage including reasoning tokens and cached tokens`. Parse usage dict; set `scope._input_tokens`, `scope._output_tokens`; from `output_tokens_details` set `scope._reasoning_tokens`; from `input_tokens_details` or `prompt_tokens_details` set `scope._cache_read_input_tokens = ... .get("cached_tokens", 0)`. Set `scope._cache_creation_input_tokens` if the API exposes it.++### 4.6 extra_attrs keys for emit_inference_event++Same keys across instrumentations: `response_id`, `finish_reasons`, `output_type`; `temperature`, `max_tokens`, `top_p`, `frequency_penalty`, `presence_penalty`, `stop_sequences`, `seed`, `choice_count` or `n`; `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens` (always pass, even when 0); `system_instructions`, `error_type`, `conversation_id` (when available).++**Flow summary:** Entrypoint → create span, call SDK → normalize response to dict → build scope, parse usage/cache/messages → call common_*_logic → (cost → common_span_attributes → request/response/tools/usage/cache → content + emit_inference_event if capture_message_content → status + metrics). Streaming: accumulate then call common_*_logic once at the end.++---++## 5. Reusing and Sharing Code++### 5.1 From `openlit.__helpers`++- **`common_span_attributes(...)`** – Use for all instrumentations so common span attributes are identical.+- **`response_as_dict(response)`** – Use when the SDK returns an object; keeps parsing in utils dict-based.+- **`otel_event(name, attributes, body)`** – Use inside your `emit_inference_event`-style helper.+- **`calculate_ttft` / `calculate_tbt`** – For streaming metrics.+- **`get_chat_model_cost` / `get_embed_model_cost`** – For cost; provider-specific pricing can live in helpers or in the provider’s utils.+- **`record_completion_metrics`** – For metrics.+- **`handle_exception(span, e)`** – Set `error.type` and span status; use when you catch exceptions in the instrumentation.++### 5.2 From `openlit.semcov`++- **`SemanticConvention`** – Use for every attribute and event name (e.g. `GEN_AI_OPERATION`, `GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS`, `GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS`). Do not hardcode OTel strings in instrumentation code.++### 5.3 Provider-specific utils++- **`build_input_messages`** / **`build_output_messages`** – Provider-specific request/response shape → OTel message array. Same return shape across providers so **`_set_span_messages_as_array`** and **`emit_inference_event`** can stay shared or easily copied.+- **`emit_inference_event`** – Can live in the provider’s `utils.py` (as in OpenAI) with the **extra_attrs** key→OTel mapping documented above. New instrumentations should reuse the same **extra** keys and the same OTel attribute mapping so events look the same everywhere.+- **Cached tokens:** In the **response parsing** step, always set **`scope._cache_read_input_tokens`** (and **`scope._cache_creation_input_tokens`** if the API exposes it), defaulting to 0. Then in common_*_logic, set the span attributes when the scope has them (even when 0), and pass the same values in **extra** so the event gets them too.++---++## 6. Summary Checklist for New Instrumentations++- [ ] Use **`common_span_attributes()`** for base span attributes.+- [ ] Set **usage attributes** on the span (input/output tokens, cost, **cache_read** and **cache_creation** even when 0).+- [ ] Use **`_set_span_messages_as_array(span, input_messages, output_messages)`** with provider-specific **`build_input_messages`** / **`build_output_messages`** so **`gen_ai.input.messages`** and **`gen_ai.output.messages`** are JSON arrays on both span and event.+- [ ] Emit one **`gen_ai.client.inference.operation.details`** event per invocation via an **`emit_inference_event`**-style helper, with **extra** including **`cache_read_input_tokens`** and **`cache_creation_input_tokens`** (even when 0).+- [ ] Map **extra** keys to **`SemanticConvention`** constants inside the event helper (no raw OTel strings in call sites).+- [ ] Parse **cached token** fields from the provider’s usage object into **scope** and pass them through to both span and event.+- [ ] Use a **scope** object with underscored attributes and a single **common_*_logic** path so streaming and non-streaming share the same attribute and event logic.++This keeps span attributes and span events consistent across providers and aligned with OTel GenAI semantic conventions while preserving a clear, reusable code structure.diff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.tomlindex ac9d3b40..951aeb31 100644--- a/sdk/python/pyproject.toml+++ b/sdk/python/pyproject.toml@@ -24,6 +24,7 @@ botocore = "^1.34.0"opentelemetry-api = "^1.30.0"opentelemetry-sdk = "^1.30.0"opentelemetry-exporter-otlp = "^1.30.0"+opentelemetry-semconv = "^0.50b0"opentelemetry-instrumentation = ">=0.52b0,<1.0.0"# Official OpenTelemetry HTTP Framework instrumentationsopentelemetry-instrumentation-asgi = ">=0.52b0,<1.0.0"diff --git a/sdk/python/src/openlit/__helpers.py b/sdk/python/src/openlit/__helpers.pyindex 5b428455..25f7d3c5 100644--- a/sdk/python/src/openlit/__helpers.py+++ b/sdk/python/src/openlit/__helpers.py@@ -174,6 +174,12 @@ def handle_exception(span, e):span.record_exception(e)span.set_status(Status(StatusCode.ERROR))+ # OTel gen-ai: conditionally required error.type (low-cardinality identifier)+ try:+ error_type = type(e).__name__ or "_OTHER"+ except Exception:+ error_type = "_OTHER"+ span.set_attribute(SemanticConvention.ERROR_TYPE, error_type)def calculate_ttft(timestamps: List[float], start_time: float) -> float:diff --git a/sdk/python/src/openlit/instrumentation/ai21/__init__.py b/sdk/python/src/openlit/instrumentation/ai21/__init__.pyindex 5664eb64..45cd1113 100644--- a/sdk/python/src/openlit/instrumentation/ai21/__init__.py+++ b/sdk/python/src/openlit/instrumentation/ai21/__init__.py@@ -27,6 +27,7 @@ class AI21Instrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("ai21")# Chat completions@@ -42,6 +43,7 @@ class AI21Instrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -74,6 +76,7 @@ class AI21Instrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/ai21/ai21.py b/sdk/python/src/openlit/instrumentation/ai21/ai21.pyindex 7d8ed572..d0340005 100644--- a/sdk/python/src/openlit/instrumentation/ai21/ai21.py+++ b/sdk/python/src/openlit/instrumentation/ai21/ai21.py@@ -26,6 +26,7 @@ def chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -55,6 +56,8 @@ def chat(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsself._start_time = time.time()@@ -98,6 +101,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:@@ -146,6 +150,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/ai21/async_ai21.py b/sdk/python/src/openlit/instrumentation/ai21/async_ai21.pyindex c6520f30..26401f42 100644--- a/sdk/python/src/openlit/instrumentation/ai21/async_ai21.py+++ b/sdk/python/src/openlit/instrumentation/ai21/async_ai21.py@@ -26,6 +26,7 @@ def async_chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -55,6 +56,8 @@ def async_chat(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsself._start_time = time.time()@@ -98,6 +101,7 @@ def async_chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:@@ -146,6 +150,7 @@ def async_chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/ai21/utils.py b/sdk/python/src/openlit/instrumentation/ai21/utils.pyindex da1ee9d8..32f5042b 100644--- a/sdk/python/src/openlit/instrumentation/ai21/utils.py+++ b/sdk/python/src/openlit/instrumentation/ai21/utils.py@@ -2,6 +2,8 @@AI21 OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -14,9 +16,207 @@ from openlit.__helpers import (get_chat_model_cost,common_span_attributes,record_completion_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+++def build_input_messages(messages):+ """+ Convert AI21 request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ structured = []+ for message in messages or []:+ if hasattr(message, "role") and (+ hasattr(message, "content") or hasattr(message, "text")+ ):+ role = (+ str(message.role.value)+ if hasattr(message.role, "value")+ else str(message.role)+ )+ content = getattr(message, "content", None) or getattr(message, "text", "")+ elif isinstance(message, dict):+ role = message.get("role", "user")+ content = message.get("content", "")+ else:+ role = str(getattr(message, "role", "user"))+ content = str(+ getattr(message, "content", "") or getattr(message, "text", "")+ )+ if isinstance(content, list):+ parts = []+ for item in content:+ if isinstance(item, dict):+ if item.get("type") == "text":+ parts.append({"type": "text", "content": item.get("text", "")})+ elif "image_url" in item:+ parts.append(+ {+ "type": "uri",+ "modality": "image",+ "uri": item.get("image_url", ""),+ }+ )+ else:+ parts.append({"type": "text", "content": str(item)})+ else:+ parts = [{"type": "text", "content": str(content)}]+ if parts:+ structured.append({"role": role, "parts": parts})+ return structured+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert AI21 response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls:+ if not isinstance(tool_calls, list):+ tool_calls = [tool_calls]+ for tc in tool_calls:+ func = tc.get("function", {}) if isinstance(tc, dict) else {}+ if isinstance(func, dict):+ parts.append(+ {+ "type": "tool_call",+ "id": tc.get("id", ""),+ "name": func.get("name", ""),+ "arguments": func.get("arguments", {}),+ }+ )+ if not parts:+ parts = [{"type": "text", "content": ""}]+ return [+ {"role": "assistant", "parts": parts, "finish_reason": finish_reason or "stop"}+ ]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return++ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port is not None:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions++ for key, value in extra_attrs.items():+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value++ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)++ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)+def format_content(messages):"""@@ -86,9 +286,17 @@ def process_chunk(scope, chunk):if content:scope._llmresponse += content+ # Handle token usage including reasoning tokens and cached tokensif chunked.get("usage"):- scope._input_tokens = chunked.get("usage").get("prompt_tokens")- scope._output_tokens = chunked.get("usage").get("completion_tokens")+ usage = chunked.get("usage", {})+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (+ usage.get("input_tokens_details") or {}+ ).get("cache_creation_tokens", 0) or 0scope._response_id = chunked.get("id")scope._finish_reason = chunked.get("choices", [{}])[0].get("finish_reason")scope._end_time = time.time()@@ -104,10 +312,12 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process chat request and generate Telemetry"""+ scope._end_time = time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)@@ -115,9 +325,19 @@ def common_chat_logic(prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "jamba-1.5-mini")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ if (+ hasattr(scope, "_input_tokens")+ and scope._input_tokens is not None+ and hasattr(scope, "_output_tokens")+ and scope._output_tokens is not None+ ):+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributescommon_span_attributes(@@ -127,7 +347,7 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- request_model,+ getattr(scope, "_response_model", request_model),environment,application_name,is_stream,@@ -164,7 +384,10 @@ def common_chat_logic()# Span Attributes for Response parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason])@@ -173,19 +396,6 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)- # Span Attributes for Cost and Tokens- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,- )- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-# Span Attributes for Toolsif scope._tools:scope._span.set_attribute(@@ -200,26 +410,77 @@ def common_chat_logic(str(scope._tools.get("function", {}).get("arguments", "")),)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)-- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },++ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, tool_calls=scope._tools)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop"),+ "seed": scope._kwargs.get("seed"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=getattr(scope, "_response_model", request_model),+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -232,13 +493,13 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- request_model,+ getattr(scope, "_response_model", request_model),environment,application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -254,6 +515,7 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process streaming chat request and generate Telemetry@@ -269,6 +531,7 @@ def process_streaming_chat_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -286,6 +549,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -304,8 +568,17 @@ def process_chat_response(for choice in response_dict.get("choices", []))scope._response_id = response_dict.get("id")- scope._input_tokens = response_dict.get("usage", {}).get("prompt_tokens", 0)- scope._output_tokens = response_dict.get("usage", {}).get("completion_tokens", 0)+ scope._response_model = response_dict.get("model") or request_model+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage", {})+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (usage.get("input_tokens_details") or {}).get(+ "cache_creation_tokens", 0+ ) or 0scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_port@@ -332,6 +605,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return response@@ -417,6 +691,18 @@ def common_chat_rag_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )+# Handle tool callsif scope._kwargs.get("tools"):scope._span.set_attribute(@@ -516,6 +802,14 @@ def process_chat_rag_response(scope._response_model = request_modelscope._input_tokens = input_tokensscope._output_tokens = output_tokens+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage", {})+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (usage.get("input_tokens_details") or {}).get(+ "cache_creation_tokens", 0+ ) or 0scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portdiff --git a/sdk/python/src/openlit/instrumentation/anthropic/anthropic.py b/sdk/python/src/openlit/instrumentation/anthropic/anthropic.pyindex 8bf1b8b7..c89704d0 100644--- a/sdk/python/src/openlit/instrumentation/anthropic/anthropic.py+++ b/sdk/python/src/openlit/instrumentation/anthropic/anthropic.py@@ -52,6 +52,8 @@ def messages(self._finish_reason = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._tool_arguments = ""self._tool_id = ""self._tool_name = ""diff --git a/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.py b/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.pyindex 1bfcd97d..f987145c 100644--- a/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.py+++ b/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.py@@ -52,6 +52,8 @@ def async_messages(self._finish_reason = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._tool_arguments = ""self._tool_id = ""self._tool_name = ""diff --git a/sdk/python/src/openlit/instrumentation/anthropic/utils.py b/sdk/python/src/openlit/instrumentation/anthropic/utils.pyindex 1e7a0869..0aaaf04a 100644--- a/sdk/python/src/openlit/instrumentation/anthropic/utils.py+++ b/sdk/python/src/openlit/instrumentation/anthropic/utils.py@@ -2,6 +2,8 @@Anthropic OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -13,9 +15,13 @@ from openlit.__helpers import (get_chat_model_cost,record_completion_metrics,common_span_attributes,+ general_tokens,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -53,6 +59,7 @@ def format_content(messages):def build_input_messages(messages, system=None):"""Convert Anthropic request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.Args:messages: Anthropic messages array from request@@ -61,10 +68,6 @@ def build_input_messages(messages, system=None):Returns:List of ChatMessage objects with role and parts"""- import logging-- logger = logging.getLogger(__name__)-if not messages:return []@@ -189,6 +192,7 @@ def build_input_messages(messages, system=None):def build_output_messages(response_text, finish_reason, tool_calls=None):"""Convert Anthropic response to OTel output message structure.+ Follows gen-ai-output-messages schema.Args:response_text: Response text from model@@ -198,10 +202,6 @@ def build_output_messages(response_text, finish_reason, tool_calls=None):Returns:List with single OutputMessage"""- import logging-- logger = logging.getLogger(__name__)-parts = []try:@@ -223,9 +223,9 @@ def build_output_messages(response_text, finish_reason, tool_calls=None):# Map Anthropic finish reasons to OTel standardfinish_reason_map = {"end_turn": "stop",- "max_tokens": "max_tokens",+ "max_tokens": "length","stop_sequence": "stop",- "tool_use": "tool_calls",+ "tool_use": "tool_call",}otel_finish_reason = finish_reason_map.get(@@ -251,10 +251,6 @@ def build_tool_definitions(tools):Returns:List of tool definition objects or None"""- import logging-- logger = logging.getLogger(__name__)-if not tools:return None@@ -296,6 +292,27 @@ def build_tool_definitions(tools):return None+def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -323,16 +340,10 @@ def emit_inference_event(server_port: Server port**extra_attrs: Additional attributes (temperature, max_tokens, etc.)"""- import logging-- logger = logging.getLogger(__name__)-try:if not event_provider:return- from openlit.__helpers import otel_event-# Build event attributesattributes = {SemanticConvention.GEN_AI_OPERATION: operation_name,@@ -391,6 +402,13 @@ def emit_inference_event(attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ else:+ # Pass through any other attributes as-is+ attributes[key] = value# Create and emit eventevent = otel_event(@@ -420,36 +438,48 @@ def process_chunk(scope, chunk):chunked = response_as_dict(chunk)- # Collect message IDs and input token from events+ # Collect message IDs and input token from events (message_start: message.usage)if chunked.get("type") == "message_start":- scope._response_id = chunked.get("message").get("id")- message_usage = chunked.get("message").get("usage", {})- scope._input_tokens = message_usage.get("input_tokens", 0)- # Extract cache tokens from message_start event- scope._cache_creation_input_tokens = message_usage.get(- "cache_creation_input_tokens", 0+ message = chunked.get("message") or {}+ message_usage = message.get("usage") or {}+ scope._response_id = message.get("id")+ scope._input_tokens = message_usage.get("input_tokens", 0) or 0+ # Extract cache tokens from message_start event (default 0 when absent)+ scope._cache_creation_input_tokens = (+ message_usage.get("cache_creation_input_tokens", 0) or 0+ )+ scope._cache_read_input_tokens = (+ message_usage.get("cache_read_input_tokens", 0) or 0)- scope._cache_read_input_tokens = message_usage.get("cache_read_input_tokens", 0)- scope._response_model = chunked.get("message").get("model")- scope._response_role = chunked.get("message").get("role")+ scope._response_model = message.get("model")+ scope._response_role = message.get("role")# Collect message IDs and aggregated response from eventsif chunked.get("type") == "content_block_delta":- if chunked.get("delta").get("text"):- scope._llmresponse += chunked.get("delta").get("text")- elif chunked.get("delta").get("partial_json"):- scope._tool_arguments += chunked.get("delta").get("partial_json")+ delta = chunked.get("delta") or {}+ if delta.get("text"):+ scope._llmresponse += delta.get("text", "")+ elif delta.get("partial_json"):+ scope._tool_arguments += delta.get("partial_json", "")if chunked.get("type") == "content_block_start":- if chunked.get("content_block").get("id"):- scope._tool_id = chunked.get("content_block").get("id")- if chunked.get("content_block").get("name"):- scope._tool_name = chunked.get("content_block").get("name")+ content_block = chunked.get("content_block") or {}+ if content_block.get("id"):+ scope._tool_id = content_block.get("id")+ if content_block.get("name"):+ scope._tool_name = content_block.get("name")- # Collect output tokens and stop reason from events+ # Collect output tokens and stop reason from events (message_delta: top-level usage, delta)if chunked.get("type") == "message_delta":- scope._output_tokens = chunked.get("usage").get("output_tokens")- scope._finish_reason = chunked.get("delta").get("stop_reason")+ usage = chunked.get("usage") or {}+ delta = chunked.get("delta") or {}+ scope._output_tokens = usage.get("output_tokens", 0) or 0+ scope._finish_reason = delta.get("stop_reason") or scope._finish_reason+ # message_delta carries final usage; update cache token counts when present+ scope._cache_creation_input_tokens = (+ usage.get("cache_creation_input_tokens", 0) or 0+ )+ scope._cache_read_input_tokens = usage.get("cache_read_input_tokens", 0) or 0def common_chat_logic(@@ -472,12 +502,18 @@ def common_chat_logic(if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- formatted_messages = format_content(scope._kwargs.get("messages", []))+ prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "claude-3-5-sonnet-latest")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ # Calculate tokens and cost+ if hasattr(scope, "_input_tokens") and scope._input_tokens is not None:+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributescommon_span_attributes(@@ -517,7 +553,10 @@ def common_chat_logic()# Span Attributes for Response parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ if scope._response_id:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason])@@ -526,54 +565,87 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if hasattr(scope, "_tool_calls") and scope._tool_calls:+ tool_name = scope._tool_calls.get("name", "")+ tool_id = scope._tool_calls.get("id", "")+ tool_args = scope._tool_calls.get("input", "")++ scope._span.set_attribute(SemanticConvention.GEN_AI_TOOL_NAME, tool_name)+ scope._span.set_attribute(SemanticConvention.GEN_AI_TOOL_CALL_ID, tool_id)+ scope._span.set_attribute(SemanticConvention.GEN_AI_TOOL_ARGS, str(tool_args))+# Span Attributes for Cost and Tokensscope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,+ input_tokens + output_tokens,)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Handle tool calls if present- if scope._tool_calls:- # Optimized tool handling - extract name, id, and arguments- tool_name = scope._tool_calls.get("name", "")- tool_id = scope._tool_calls.get("id", "")- tool_args = scope._tool_calls.get("input", "")-- scope._span.set_attribute(SemanticConvention.GEN_AI_TOOL_NAME, tool_name)- scope._span.set_attribute(SemanticConvention.GEN_AI_TOOL_CALL_ID, tool_id)- scope._span.set_attribute(SemanticConvention.GEN_AI_TOOL_ARGS, str(tool_args))-- # Span Attributes for Content- if capture_message_content:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_INPUT_MESSAGES, formatted_messages+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(+ scope._kwargs.get("messages", []),+ system=scope._kwargs.get("system"),+ )+ output_msgs = build_output_messages(+ scope._llmresponse,+ scope._finish_reason,+ scope._tool_calls if hasattr(scope, "_tool_calls") else None,+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ # gen_ai.system_instructions (Anthropic: from request system)+ system_raw = scope._kwargs.get("system")+ if system_raw:+ system_instr = [{"type": "text", "content": system_raw}]+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS,+ json.dumps(system_instr),+ )+# Emit inference eventif event_provider:- import logging-- logger = logging.getLogger(__name__)try:- input_msgs = build_input_messages(- scope._kwargs.get("messages", []),- system=scope._kwargs.get("system"),- )- output_msgs = build_output_messages(- scope._llmresponse, scope._finish_reason, scope._tool_calls- )tool_defs = build_tool_definitions(scope._kwargs.get("tools"))-+ output_type = "text" if isinstance(scope._llmresponse, str) else "json"+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": output_type,+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "top_k": scope._kwargs.get("top_k"),+ "stop_sequences": scope._kwargs.get("stop_sequences"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ if system_raw:+ extra["system_instructions"] = system_instremit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,@@ -584,23 +656,7 @@ def common_chat_logic(tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- response_id=scope._response_id,- finish_reasons=[scope._finish_reason],- temperature=scope._kwargs.get("temperature"),- max_tokens=scope._kwargs.get("max_tokens"),- top_p=scope._kwargs.get("top_p"),- top_k=scope._kwargs.get("top_k"),- stop_sequences=scope._kwargs.get("stop_sequences"),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,- cache_creation_input_tokens=scope._cache_creation_input_tokens- if hasattr(scope, "_cache_creation_input_tokens")- and scope._cache_creation_input_tokens > 0- else None,- cache_read_input_tokens=scope._cache_read_input_tokens- if hasattr(scope, "_cache_read_input_tokens")- and scope._cache_read_input_tokens > 0- else None,+ **extra,)except Exception as e:logger.warning("Failed to emit inference event: %s", e, exc_info=True)@@ -621,8 +677,8 @@ def common_chat_logic(application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -696,7 +752,7 @@ def process_chat_response(scope._llmresponse = response_dict.get("content", [{}])[0].get("text", "")scope._response_role = response_dict.get("role", "assistant")- # Extract usage including cache tokens (Anthropic prompt caching)+ # Handle token usage including reasoning tokens and cached tokensusage = response_dict.get("usage", {})scope._input_tokens = usage.get("input_tokens", 0)scope._output_tokens = usage.get("output_tokens", 0)diff --git a/sdk/python/src/openlit/instrumentation/assemblyai/__init__.py b/sdk/python/src/openlit/instrumentation/assemblyai/__init__.pyindex b2df2b46..730fb65a 100644--- a/sdk/python/src/openlit/instrumentation/assemblyai/__init__.py+++ b/sdk/python/src/openlit/instrumentation/assemblyai/__init__.py@@ -26,6 +26,7 @@ class AssemblyAIInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("assemblyai")# sync transcribe@@ -41,6 +42,7 @@ class AssemblyAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/assemblyai/assemblyai.py b/sdk/python/src/openlit/instrumentation/assemblyai/assemblyai.pyindex 63ca0b03..b603990e 100644--- a/sdk/python/src/openlit/instrumentation/assemblyai/assemblyai.py+++ b/sdk/python/src/openlit/instrumentation/assemblyai/assemblyai.py@@ -22,6 +22,7 @@ def transcribe(capture_message_content,metrics,disable_metrics,+ event_provider,):"""Generates a telemetry wrapper for AssemblyAI transcribe function call@@ -47,7 +48,6 @@ def transcribe(response = process_audio_response(response=response,- gen_ai_endpoint="assemblyai.transcribe",pricing_info=pricing_info,server_port=server_port,server_address=server_address,@@ -59,6 +59,7 @@ def transcribe(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/assemblyai/utils.py b/sdk/python/src/openlit/instrumentation/assemblyai/utils.pyindex 16e9f54c..f1baed07 100644--- a/sdk/python/src/openlit/instrumentation/assemblyai/utils.py+++ b/sdk/python/src/openlit/instrumentation/assemblyai/utils.py@@ -2,21 +2,22 @@AssemblyAI OpenTelemetry instrumentation utility functions"""+import json+import loggingimport time-from opentelemetry.sdk.resources import (- SERVICE_NAME,- TELEMETRY_SDK_NAME,- DEPLOYMENT_ENVIRONMENT,-)from opentelemetry.trace import Status, StatusCodefrom openlit.__helpers import (+ common_span_attributes,get_audio_model_cost,- create_metrics_attributes,+ otel_event,+ record_audio_metrics,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_audio_url(audio_url):"""@@ -25,80 +26,172 @@ def format_audio_url(audio_url):return str(audio_url) if audio_url else ""-def common_span_attributes(- scope,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,- request_model,- response_model,- environment,- application_name,- is_stream,- tbt,- ttft,- version,-):+def format_content(messages):"""- Set common span attributes for both chat and RAG operations.+ Format the request input for display/cost. For AssemblyAI, input is typically+ an audio URL string; pass-through for consistency with other instrumentations."""+ if messages is None:+ return ""+ if isinstance(messages, str):+ return messages+ return str(messages)- scope._span.set_attribute(TELEMETRY_SDK_NAME, "openlit")- scope._span.set_attribute(SemanticConvention.GEN_AI_OPERATION, gen_ai_operation)- scope._span.set_attribute(- SemanticConvention.GEN_AI_PROVIDER_NAME, GEN_AI_PROVIDER_NAME- )- scope._span.set_attribute(SemanticConvention.SERVER_ADDRESS, server_address)- scope._span.set_attribute(SemanticConvention.SERVER_PORT, server_port)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, request_model)- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_MODEL, scope._response_model++def build_input_messages(audio_url_or_request):+ """+ Convert AssemblyAI request to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not audio_url_or_request:+ return []+ content = (+ format_audio_url(audio_url_or_request)+ if isinstance(audio_url_or_request, str)+ else str(audio_url_or_request))- scope._span.set_attribute(DEPLOYMENT_ENVIRONMENT, environment)- scope._span.set_attribute(SERVICE_NAME, application_name)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, is_stream)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TBT, scope._tbt)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TTFT, scope._ttft)- scope._span.set_attribute(SemanticConvention.GEN_AI_SDK_VERSION, version)+ if not content:+ return []+ return [{"role": "user", "parts": [{"type": "text", "content": content}]}]-def record_audio_metrics(- metrics,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,+def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert AssemblyAI response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ content = str(response_text) if response_text is not None else ""+ msg = {+ "role": "assistant",+ "parts": [{"type": "text", "content": content}],+ "finish_reason": finish_reason or "stop",+ }+ return [msg]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,request_model,response_model,- environment,- application_name,- start_time,- end_time,- cost,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,):"""- Record audio metrics for the operation.+ Emit gen_ai.client.inference.operation.details event."""+ try:+ if not event_provider:+ return- attributes = create_metrics_attributes(- operation=gen_ai_operation,- system=GEN_AI_PROVIDER_NAME,- server_address=server_address,- server_port=server_port,- request_model=request_model,- response_model=response_model,- service_name=application_name,- deployment_environment=environment,- )- metrics["genai_client_operation_duration"].record(end_time - start_time, attributes)- metrics["genai_requests"].add(1, attributes)- metrics["genai_cost"].record(cost, attributes)+ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions++ for key, value in extra_attrs.items():+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value++ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)++ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)def common_audio_logic(scope,- gen_ai_endpoint,pricing_info,environment,application_name,@@ -106,16 +199,17 @@ def common_audio_logic(capture_message_content,disable_metrics,version,+ is_stream,+ event_provider=None,):"""- Process audio transcription request and generate Telemetry+ Process audio transcription request and generate Telemetry."""prompt = scope._response.audio_urlrequest_model = scope._kwargs.get("speech_model", "best")- is_stream = False- # Calculate cost based on audio duration+ # Compute costcost = get_audio_model_cost(request_model, pricing_info, prompt, scope._response.audio_duration)@@ -128,7 +222,7 @@ def common_audio_logic(scope._server_address,scope._server_port,request_model,- request_model,+ scope._response_model,environment,application_name,is_stream,@@ -137,42 +231,86 @@ def common_audio_logic(version,)+ # AssemblyAI transcription (no additional API type attribute)++ # Span Attributes for Request parameters+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_AUDIO_DURATION,+ scope._response.audio_duration,+ )+# Span Attributes for Response parametersscope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, "text")scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response.id)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, ["stop"]+ )- # Span Attributes for Cost- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Audio-specific span attributes+ # Span Attributes for Cost and Tokens+ input_tokens = getattr(scope, "_input_tokens", 0)+ output_tokens = getattr(scope, "_output_tokens", 0)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens+ )scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_AUDIO_DURATION, scope._response.audio_duration+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._response.text- )-- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._response.text,- },+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- scope._span.set_status(Status(StatusCode.OK))+ # Span Attributes for Content+ if capture_message_content:+ input_msgs = build_input_messages(prompt)+ output_msgs = build_output_messages(scope._response.text, "stop")+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ extra = {+ "response_id": scope._response.id,+ "finish_reasons": ["stop"],+ "output_type": "text",+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_AUDIO,+ request_model=request_model,+ response_model=scope._response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)- # Metrics+ # Span status and metrics+ scope._span.set_status(Status(StatusCode.OK))if not disable_metrics:record_audio_metrics(metrics,@@ -192,7 +330,6 @@ def common_audio_logic(def process_audio_response(response,- gen_ai_endpoint,pricing_info,server_port,server_address,@@ -204,10 +341,11 @@ def process_audio_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""- Process audio transcription request and generate Telemetry+ Process audio transcription request and generate Telemetry."""scope = type("GenericScope", (), {})()@@ -219,14 +357,28 @@ def process_audio_response(scope._kwargs = kwargsscope._response = response- # Initialize streaming and timing values for AssemblyAI transcriptionscope._response_model = kwargs.get("speech_model", "best")scope._tbt = 0.0scope._ttft = scope._end_time - scope._start_time+ # Handle token usage including reasoning tokens and cached tokens+ usage = getattr(response, "usage", None)+ if isinstance(usage, dict):+ scope._input_tokens = usage.get("input_tokens", 0)+ scope._output_tokens = usage.get("output_tokens", 0)+ input_details = usage.get("input_tokens_details") or usage.get(+ "prompt_tokens_details", {}+ )+ scope._cache_read_input_tokens = input_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = usage.get("cache_creation_input_tokens", 0)+ else:+ scope._input_tokens = 0+ scope._output_tokens = 0+ scope._cache_read_input_tokens = 0+ scope._cache_creation_input_tokens = 0+common_audio_logic(scope,- gen_ai_endpoint,pricing_info,environment,application_name,@@ -234,6 +386,8 @@ def process_audio_response(capture_message_content,disable_metrics,version,+ is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/azure_ai_inference/async_azure_ai_inference.py b/sdk/python/src/openlit/instrumentation/azure_ai_inference/async_azure_ai_inference.pyindex 764321bd..93f79007 100644--- a/sdk/python/src/openlit/instrumentation/azure_ai_inference/async_azure_ai_inference.py+++ b/sdk/python/src/openlit/instrumentation/azure_ai_inference/async_azure_ai_inference.py@@ -64,6 +64,8 @@ def async_complete(self._input_tokens = 0self._output_tokens = 0self._reasoning_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsdiff --git a/sdk/python/src/openlit/instrumentation/azure_ai_inference/azure_ai_inference.py b/sdk/python/src/openlit/instrumentation/azure_ai_inference/azure_ai_inference.pyindex 9e80d773..704d35d3 100644--- a/sdk/python/src/openlit/instrumentation/azure_ai_inference/azure_ai_inference.py+++ b/sdk/python/src/openlit/instrumentation/azure_ai_inference/azure_ai_inference.py@@ -64,6 +64,8 @@ def complete(self._input_tokens = 0self._output_tokens = 0self._reasoning_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsdiff --git a/sdk/python/src/openlit/instrumentation/azure_ai_inference/utils.py b/sdk/python/src/openlit/instrumentation/azure_ai_inference/utils.pyindex d9d2ce8c..230fd479 100644--- a/sdk/python/src/openlit/instrumentation/azure_ai_inference/utils.py+++ b/sdk/python/src/openlit/instrumentation/azure_ai_inference/utils.py@@ -2,6 +2,8 @@Azure AI Inference OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -15,10 +17,13 @@ from openlit.__helpers import (common_span_attributes,record_completion_metrics,record_embedding_metrics,+ general_tokens,otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -47,6 +52,7 @@ def format_content(messages):def build_input_messages(messages):"""Convert Azure AI Inference messages to OTel input message structure.+ Follows gen-ai-input-messages schema.Args:messages: List of message objects (OpenAI-compatible format)@@ -111,6 +117,7 @@ def build_input_messages(messages):def build_output_messages(response_text, finish_reason, tool_calls=None):"""Convert Azure AI Inference response to OTel output message structure.+ Follows gen-ai-output-messages schema.Args:response_text: The text response@@ -166,6 +173,57 @@ def build_tool_definitions(tools):return tools if tools else None+def build_system_instructions_from_messages(messages):+ """+ Extract system message(s) from chat messages for gen_ai.system_instructions.+ Returns list of { type: "text", content: "..." } per OTel schema, or None.+ """+ if not messages:+ return None+ instructions = []+ for msg in messages:+ role = (+ msg.get("role", "") if isinstance(msg, dict) else getattr(msg, "role", "")+ )+ if role != "system":+ continue+ content = (+ msg.get("content", "")+ if isinstance(msg, dict)+ else getattr(msg, "content", "")+ )+ if isinstance(content, list):+ for part in content:+ if isinstance(part, dict) and part.get("type") == "text":+ text = part.get("text", "")+ if text:+ instructions.append({"type": "text", "content": text})+ elif content:+ instructions.append({"type": "text", "content": str(content)})+ return instructions if instructions else None+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -197,72 +255,92 @@ def emit_inference_event(if not event_provider:return- # Build base attributesattributes = {- SemanticConvention.GEN_AI_OPERATION_NAME: operation_name,+ SemanticConvention.GEN_AI_OPERATION: operation_name,}-- # Add model attributesif request_model:attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_modelif response_model:attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model-- # Add server attributesif server_address:- attributes["server.address"] = server_address+ attributes[SemanticConvention.SERVER_ADDRESS] = server_addressif server_port:- attributes["server.port"] = server_port-- # Add messages (only if not None/empty)- if input_messages:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages- if output_messages:+ if output_messages is not None:attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages-- # Add tool definitions (recommended)- if tool_definitions:+ if tool_definitions is not None:attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions- # Map extra attributes to semantic conventions- attr_mapping = {- "temperature": SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- "max_tokens": SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,- "top_p": SemanticConvention.GEN_AI_REQUEST_TOP_P,- "frequency_penalty": SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- "presence_penalty": SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,- "stop_sequences": SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,- "seed": SemanticConvention.GEN_AI_REQUEST_SEED,- "response_id": SemanticConvention.GEN_AI_RESPONSE_ID,- "finish_reasons": SemanticConvention.GEN_AI_RESPONSE_FINISH_REASONS,- "choice_count": SemanticConvention.GEN_AI_RESPONSE_CHOICE_COUNT,- "output_type": SemanticConvention.GEN_AI_RESPONSE_OUTPUT_TYPE,- "input_tokens": SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,- "output_tokens": SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,- "reasoning_tokens": "gen_ai.usage.reasoning_tokens", # Azure-specific- "encoding_format": "gen_ai.request.encoding_format", # For embeddings- "dimensions": "gen_ai.request.dimensions", # For embeddings- "cache_creation_input_tokens": SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,- "cache_read_input_tokens": SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,- }-for key, value in extra_attrs.items():- if value is not None and key in attr_mapping:- attributes[attr_mapping[key]] = value+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "reasoning_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_REASONING_TOKENS] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "encoding_format":+ attributes[SemanticConvention.GEN_AI_REQUEST_ENCODING_FORMATS] = (+ [value] if not isinstance(value, list) else value+ )+ elif key == "dimensions":+ attributes[SemanticConvention.GEN_AI_EMBEDDINGS_DIMENSION_COUNT] = (+ value+ )- # Create and emit eventevent = otel_event(name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,attributes=attributes,- body="", # Per spec, all data in attributes+ body="",)-event_provider.emit(event)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)@@ -332,18 +410,22 @@ def process_chunk(scope, chunk):"arguments", "")- # Handle usage information (typically only in final chunk)+ # Handle token usage including reasoning tokens and cached tokensif chunked.get("usage"):- scope._input_tokens = chunked.get("usage").get("prompt_tokens", 0)- scope._output_tokens = chunked.get("usage").get("completion_tokens", 0)- # Handle reasoning tokens if present (optional) - check nested structure- completion_details = chunked.get("usage", {}).get(- "completion_tokens_details", {}- )+ usage = chunked.get("usage", {})+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (+ usage.get("input_tokens_details") or {}+ ).get("cache_creation_tokens", 0) or 0+ completion_details = usage.get("completion_tokens_details", {})if "reasoning_tokens" in completion_details:scope._reasoning_tokens = completion_details.get("reasoning_tokens", 0)- elif "reasoning_tokens" in chunked.get("usage", {}):- scope._reasoning_tokens = chunked.get("usage").get("reasoning_tokens", 0)+ elif "reasoning_tokens" in usage:+ scope._reasoning_tokens = usage.get("reasoning_tokens", 0)def common_chat_logic(@@ -361,6 +443,7 @@ def common_chat_logic("""Process chat request and generate Telemetry"""+ scope._end_time = time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)@@ -368,9 +451,19 @@ def common_chat_logic(prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "gpt-4o")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ if (+ hasattr(scope, "_input_tokens")+ and scope._input_tokens is not None+ and hasattr(scope, "_output_tokens")+ and scope._output_tokens is not None+ ):+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributescommon_span_attributes(@@ -414,47 +507,29 @@ def common_chat_logic()# Span Attributes for Response parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ if scope._response_id:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason])- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_SERVICE_TIER, scope._response_service_tier- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_SYSTEM_FINGERPRINT,- scope._response_service_tier,- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_TYPE,- "text" if isinstance(scope._llmresponse, str) else "json",- )-- # Span Attributes for Cost and Tokens- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,- )- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Span Attributes for Reasoning (if present)- if hasattr(scope, "_reasoning_tokens") and scope._reasoning_tokens > 0:+ if getattr(scope, "_response_service_tier", None):scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_REASONING_TOKENS, scope._reasoning_tokens+ SemanticConvention.GEN_AI_RESPONSE_SERVICE_TIER,+ scope._response_service_tier,)- # Update total token usage to include reasoning tokens+ if getattr(scope, "_response_service_tier", None):scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens + scope._reasoning_tokens,+ SemanticConvention.GEN_AI_RESPONSE_SYSTEM_FINGERPRINT,+ scope._response_service_tier,)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_TYPE,+ "text" if isinstance(scope._llmresponse, str) else "json",+ )- # Span Attributes for Tools - optimized+ # Span Attributes for Toolsif scope._tools:tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]@@ -484,58 +559,113 @@ def common_chat_logic(SemanticConvention.GEN_AI_TOOL_ARGS, ", ".join(filter(None, args)))- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # Span Attributes for Reasoning (if present)+ if hasattr(scope, "_reasoning_tokens") and scope._reasoning_tokens > 0:scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_REASONING_TOKENS, scope._reasoning_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens + scope._reasoning_tokens,+ )++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # Add reasoning content if available+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, tool_calls=scope._tools+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+if hasattr(scope, "_reasoning_content") and scope._reasoning_content:scope._span.set_attribute(SemanticConvention.GEN_AI_CONTENT_REASONING, scope._reasoning_content)- # Emit OTel log event+ system_instr = build_system_instructions_from_messages(+ scope._kwargs.get("messages", [])+ )+ if system_instr:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS, json.dumps(system_instr)+ )+if event_provider:try:- # Build structured messages- input_msgs = build_input_messages(scope._kwargs.get("messages", []))- output_msgs = build_output_messages(- scope._llmresponse, scope._finish_reason, tool_calls=scope._tools- )tool_defs = build_tool_definitions(scope._kwargs.get("tools"))-+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop"),+ "seed": scope._kwargs.get("seed"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ if (+ hasattr(scope, "_reasoning_tokens")+ and scope._reasoning_tokens is not None+ ):+ extra["reasoning_tokens"] = scope._reasoning_tokens+ if system_instr:+ extra["system_instructions"] = system_instremit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,request_model=request_model,response_model=scope._response_model,- input_messages=input_msgs,- output_messages=output_msgs,+ input_messages=build_input_messages(+ scope._kwargs.get("messages", [])+ ),+ output_messages=build_output_messages(+ scope._llmresponse,+ scope._finish_reason,+ tool_calls=scope._tools,+ ),tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- response_id=scope._response_id,- finish_reasons=[scope._finish_reason],- temperature=scope._kwargs.get("temperature"),- max_tokens=scope._kwargs.get("max_tokens"),- top_p=scope._kwargs.get("top_p"),- frequency_penalty=scope._kwargs.get("frequency_penalty"),- presence_penalty=scope._kwargs.get("presence_penalty"),- stop_sequences=scope._kwargs.get("stop"),- seed=scope._kwargs.get("seed"),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,- reasoning_tokens=scope._reasoning_tokens- if hasattr(scope, "_reasoning_tokens")- else None,+ **extra,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -554,8 +684,8 @@ def common_chat_logic(application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -632,12 +762,17 @@ def process_chat_response(if reasoning_content:scope._reasoning_content = reasoning_content- scope._input_tokens = response_dict.get("usage", {}).get("prompt_tokens", 0)- scope._output_tokens = response_dict.get("usage", {}).get("completion_tokens", 0)- # Handle reasoning tokens if present (optional) - check nested structure- completion_details = response_dict.get("usage", {}).get(- "completion_tokens_details", {}- )+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage", {})+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (usage.get("input_tokens_details") or {}).get(+ "cache_creation_tokens", 0+ ) or 0+ completion_details = usage.get("completion_tokens_details", {})if "reasoning_tokens" in completion_details:scope._reasoning_tokens = completion_details.get("reasoning_tokens", 0)elif "reasoning_tokens" in response_dict.get("usage", {}):@@ -771,9 +906,6 @@ def common_embedding_logic(dimensions=scope._kwargs.get("dimensions"),)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))diff --git a/sdk/python/src/openlit/instrumentation/bedrock/bedrock.py b/sdk/python/src/openlit/instrumentation/bedrock/bedrock.pyindex 3fd6368c..5dbf6b58 100644--- a/sdk/python/src/openlit/instrumentation/bedrock/bedrock.py+++ b/sdk/python/src/openlit/instrumentation/bedrock/bedrock.py@@ -139,6 +139,9 @@ def converse_stream(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0+ self._llm_config = kwargs.get("inferenceConfig", {})self._args = argsself._kwargs = kwargs@@ -186,7 +189,6 @@ def converse_stream(return chunkexcept StopIteration:try:- llm_config = self._kwargs.get("inferenceConfig", {})with tracer.start_as_current_span(self._span_name, kind=SpanKind.CLIENT) as self._span:@@ -199,7 +201,6 @@ def converse_stream(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,- llm_config=llm_config,event_provider=self._event_provider,)diff --git a/sdk/python/src/openlit/instrumentation/bedrock/utils.py b/sdk/python/src/openlit/instrumentation/bedrock/utils.pyindex 3e29afb3..ef23cd65 100644--- a/sdk/python/src/openlit/instrumentation/bedrock/utils.py+++ b/sdk/python/src/openlit/instrumentation/bedrock/utils.py@@ -2,6 +2,8 @@AWS Bedrock OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -14,10 +16,13 @@ from openlit.__helpers import (record_completion_metrics,common_span_attributes,handle_exception,+ general_tokens,otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -59,6 +64,7 @@ def format_content(messages):def build_input_messages(messages):"""Convert Bedrock messages to OTel input message structure.+ Follows gen-ai-input-messages schema.Args:messages: List of Bedrock message objects or dicts@@ -132,6 +138,7 @@ def build_input_messages(messages):def build_output_messages(response_text, finish_reason, tool_calls=None):"""Convert Bedrock response to OTel output message structure.+ Follows gen-ai-output-messages schema.Args:response_text: The text response from Bedrock@@ -192,6 +199,27 @@ def build_tool_definitions(tools):return None+def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -223,112 +251,125 @@ def emit_inference_event(if not event_provider:return- # Build base attributesattributes = {- SemanticConvention.GEN_AI_OPERATION_NAME: operation_name,+ SemanticConvention.GEN_AI_OPERATION: operation_name,}-- # Add model attributesif request_model:attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_modelif response_model:attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model-- # Add server attributesif server_address:- attributes["server.address"] = server_address+ attributes[SemanticConvention.SERVER_ADDRESS] = server_addressif server_port:- attributes["server.port"] = server_port-- # Add messages (only if not None/empty)- if input_messages:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages- if output_messages:+ if output_messages is not None:attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages-- # Add tool definitions (recommended)- if tool_definitions:+ if tool_definitions is not None:attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions- # Map extra attributes to semantic conventions- attr_mapping = {- "temperature": SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- "max_tokens": SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,- "top_p": SemanticConvention.GEN_AI_REQUEST_TOP_P,- "top_k": SemanticConvention.GEN_AI_REQUEST_TOP_K,- "frequency_penalty": SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- "presence_penalty": SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,- "stop_sequences": SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,- "seed": SemanticConvention.GEN_AI_REQUEST_SEED,- "response_id": SemanticConvention.GEN_AI_RESPONSE_ID,- "finish_reasons": SemanticConvention.GEN_AI_RESPONSE_FINISH_REASONS,- "choice_count": SemanticConvention.GEN_AI_RESPONSE_CHOICE_COUNT,- "output_type": SemanticConvention.GEN_AI_RESPONSE_OUTPUT_TYPE,- "input_tokens": SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,- "output_tokens": SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,- "cache_creation_input_tokens": SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,- "cache_read_input_tokens": SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,- }-for key, value in extra_attrs.items():- if value is not None and key in attr_mapping:- attributes[attr_mapping[key]] = value+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "top_k":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_K] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ else:+ attributes[key] = value- # Create and emit eventevent = otel_event(name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,attributes=attributes,- body="", # Per spec, all data in attributes+ body="",)-event_provider.emit(event)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)-def process_chunk(self, chunk):+def process_chunk(scope, chunk):"""Process a chunk of response data and update state."""end_time = time.time()- # Record the timestamp for the current chunk- self._timestamps.append(end_time)+ scope._timestamps.append(end_time)- if len(self._timestamps) == 1:- # Calculate time to first chunk- self._ttft = calculate_ttft(self._timestamps, self._start_time)+ if len(scope._timestamps) == 1:+ scope._ttft = calculate_ttft(scope._timestamps, scope._start_time)chunked = response_as_dict(chunk)- # Handle Bedrock messageStart eventif "messageStart" in chunked:message_start = chunked.get("messageStart", {})- self._response_role = message_start.get("role", "assistant")+ scope._response_role = message_start.get("role", "assistant")- # Handle Bedrock contentBlockDelta eventif "contentBlockDelta" in chunked:content_delta = chunked.get("contentBlockDelta", {})delta = content_delta.get("delta", {})if "text" in delta:- self._llmresponse += delta.get("text", "")+ scope._llmresponse += delta.get("text", "")- # Handle Bedrock messageStop eventif "messageStop" in chunked:message_stop = chunked.get("messageStop", {})- self._finish_reason = message_stop.get("stopReason", "")+ scope._finish_reason = message_stop.get("stopReason", "")- # Handle Bedrock metadata event (final event with usage info)+ # Handle token usage including reasoning tokens and cached tokensif "metadata" in chunked:- metadata = chunked.get("metadata", {})- usage = metadata.get("usage", {})- self._input_tokens = usage.get("inputTokens", 0)- self._output_tokens = usage.get("outputTokens", 0)- self._end_time = end_time+ metadata = chunked.get("metadata") or {}+ usage = metadata.get("usage") or {}+ scope._input_tokens = usage.get("inputTokens", 0) or 0+ scope._output_tokens = usage.get("outputTokens", 0) or 0+ scope._cache_read_input_tokens = usage.get("cacheReadInputTokens", 0) or 0+ scope._cache_creation_input_tokens = usage.get("cacheWriteInputTokens", 0) or 0+ scope._end_time = end_timedef common_chat_logic(@@ -340,7 +381,6 @@ def common_chat_logic(capture_message_content,disable_metrics,version,- llm_config,is_stream,event_provider=None,):@@ -352,12 +392,19 @@ def common_chat_logic(if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- formatted_messages = format_content(scope._kwargs.get("messages", []))+ llm_config = getattr(scope, "_llm_config", None) or {}+ prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("modelId", "amazon.titan-text-express-v1")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ # Calculate tokens and cost+ if hasattr(scope, "_input_tokens") and scope._input_tokens is not None:+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributescommon_span_attributes(@@ -376,7 +423,7 @@ def common_chat_logic(version,)- # Bedrock-specific attributes from llm_config+ # Span Attributes for Request parameters (Bedrock inferenceConfig)bedrock_attributes = [(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, "frequencyPenalty"),(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, "maxTokens"),@@ -386,15 +433,20 @@ def common_chat_logic((SemanticConvention.GEN_AI_REQUEST_TOP_P, "topP"),(SemanticConvention.GEN_AI_REQUEST_TOP_K, "topK"),]-- # Set each bedrock-specific attribute if the corresponding value exists and is not Nonefor attribute, key in bedrock_attributes:- value = llm_config.get(key)+ value = (+ llm_config.get(key)+ if isinstance(llm_config, dict)+ else getattr(llm_config, key, None)+ )if value is not None:scope._span.set_attribute(attribute, value)# Span Attributes for Response parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason])@@ -403,46 +455,104 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if hasattr(scope, "_tools") and scope._tools:+ tools = scope._tools if isinstance(scope._tools, dict) else {}+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, tools.get("name", "")+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, str(tools.get("id", ""))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS, str(tools.get("args", ""))+ )+# Span Attributes for Cost and Tokensscope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content- if capture_message_content:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_INPUT_MESSAGES, formatted_messages+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # Emit OTel log event+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse,+ scope._finish_reason,+ tool_calls=None,+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ # gen_ai.system_instructions (Bedrock: from system block in request when present)+ system_parts = []+ system_block = scope._kwargs.get("system") or []+ if system_block and isinstance(system_block, list):+ for item in system_block:+ if isinstance(item, dict) and item.get("text"):+ system_parts.append({"type": "text", "content": item["text"]})+ if system_parts:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS,+ json.dumps(system_parts),+ )+if event_provider:try:- # Build structured messages- input_msgs = build_input_messages(scope._kwargs.get("messages", []))- output_msgs = build_output_messages(- scope._llmresponse,- scope._finish_reason,- tool_calls=None, # No tool support yet- )- # pylint: disable=assignment-from-none- tool_defs = build_tool_definitions(- scope._kwargs.get("toolConfig", {}).get("tools")+ tool_defs = (+ build_tool_definitions(+ (scope._kwargs.get("toolConfig") or {}).get("tools")+ )+ or [])-- # Extract inferenceConfig parameters- inference_config = llm_config or {}-+ output_type = "text" if isinstance(scope._llmresponse, str) else "json"+ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": output_type,+ "temperature": llm_config.get("temperature")+ if isinstance(llm_config, dict)+ else getattr(llm_config, "temperature", None),+ "max_tokens": llm_config.get("maxTokens")+ if isinstance(llm_config, dict)+ else getattr(llm_config, "maxTokens", None),+ "top_p": llm_config.get("topP")+ if isinstance(llm_config, dict)+ else getattr(llm_config, "topP", None),+ "top_k": llm_config.get("topK")+ if isinstance(llm_config, dict)+ else getattr(llm_config, "topK", None),+ "stop_sequences": llm_config.get("stopSequences")+ if isinstance(llm_config, dict)+ else getattr(llm_config, "stopSequences", None),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ if system_parts:+ extra["system_instructions"] = system_partsemit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,@@ -453,20 +563,9 @@ def common_chat_logic(tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- response_id=scope._response_id if scope._response_id else None,- finish_reasons=[scope._finish_reason],- temperature=inference_config.get("temperature"),- max_tokens=inference_config.get("maxTokens"),- top_p=inference_config.get("topP"),- top_k=inference_config.get("topK"),- stop_sequences=inference_config.get("stopSequences"),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,+ **extra,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -485,8 +584,8 @@ def common_chat_logic(application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -502,7 +601,6 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",- llm_config=None,event_provider=None,):"""@@ -510,9 +608,6 @@ def process_streaming_chat_response("""try:- if llm_config is None:- llm_config = {}-common_chat_logic(scope,pricing_info,@@ -522,7 +617,6 @@ def process_streaming_chat_response(capture_message_content,disable_metrics,version,- llm_config,is_stream=True,event_provider=event_provider,)@@ -572,8 +666,12 @@ def process_chat_response(scope._response_role = (response_dict.get("output", {}).get("message", {}).get("role", "assistant"))- scope._input_tokens = response_dict.get("usage", {}).get("inputTokens", 0)- scope._output_tokens = response_dict.get("usage", {}).get("outputTokens", 0)+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage") or {}+ scope._input_tokens = usage.get("inputTokens", 0) or 0+ scope._output_tokens = usage.get("outputTokens", 0) or 0+ scope._cache_read_input_tokens = usage.get("cacheReadInputTokens", 0) or 0+ scope._cache_creation_input_tokens = usage.get("cacheWriteInputTokens", 0) or 0scope._response_model = request_modelscope._finish_reason = response_dict.get("stopReason", "")scope._response_id = response_dict.get("RequestId", "")@@ -581,6 +679,7 @@ def process_chat_response(scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portscope._kwargs = kwargs+ scope._llm_config = llm_config or {}common_chat_logic(scope,@@ -591,7 +690,6 @@ def process_chat_response(capture_message_content,disable_metrics,version,- llm_config,is_stream=False,event_provider=event_provider,)diff --git a/sdk/python/src/openlit/instrumentation/cohere/__init__.py b/sdk/python/src/openlit/instrumentation/cohere/__init__.pyindex a700225a..1a555285 100644--- a/sdk/python/src/openlit/instrumentation/cohere/__init__.py+++ b/sdk/python/src/openlit/instrumentation/cohere/__init__.py@@ -31,6 +31,7 @@ class CohereInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("cohere")# sync chat completions@@ -46,6 +47,7 @@ class CohereInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -62,6 +64,7 @@ class CohereInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -94,6 +97,7 @@ class CohereInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -110,6 +114,7 @@ class CohereInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/cohere/async_cohere.py b/sdk/python/src/openlit/instrumentation/cohere/async_cohere.pyindex d3ac1708..1abd76c3 100644--- a/sdk/python/src/openlit/instrumentation/cohere/async_cohere.py+++ b/sdk/python/src/openlit/instrumentation/cohere/async_cohere.py@@ -26,6 +26,7 @@ def async_chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI chat function call@@ -60,6 +61,7 @@ def async_chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)@@ -77,6 +79,7 @@ def async_chat_stream(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI chat_stream function call@@ -95,6 +98,7 @@ def async_chat_stream(kwargs,server_address,server_port,+ event_provider=None,**args,):self.__wrapped__ = wrapped@@ -108,6 +112,9 @@ def async_chat_stream(self._tool_plan = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0+ self._event_provider = event_providerself._args = argsself._kwargs = kwargs@@ -152,6 +159,7 @@ def async_chat_stream(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=self._event_provider,)except Exception as e:@@ -176,7 +184,13 @@ def async_chat_stream(span = tracer.start_span(span_name, kind=SpanKind.CLIENT)return TracedAsyncStream(- awaited_wrapped, span, span_name, kwargs, server_address, server_port+ awaited_wrapped,+ span,+ span_name,+ kwargs,+ server_address,+ server_port,+ event_provider=event_provider,)return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/cohere/cohere.py b/sdk/python/src/openlit/instrumentation/cohere/cohere.pyindex eeda7363..2b102987 100644--- a/sdk/python/src/openlit/instrumentation/cohere/cohere.py+++ b/sdk/python/src/openlit/instrumentation/cohere/cohere.py@@ -26,6 +26,7 @@ def chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI chat function call@@ -60,6 +61,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)@@ -77,6 +79,7 @@ def chat_stream(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI chat_stream function call@@ -95,6 +98,7 @@ def chat_stream(kwargs,server_address,server_port,+ event_provider=None,**args,):self.__wrapped__ = wrapped@@ -108,6 +112,9 @@ def chat_stream(self._tool_plan = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0+ self._event_provider = event_providerself._args = argsself._kwargs = kwargs@@ -152,6 +159,7 @@ def chat_stream(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=self._event_provider,)except Exception as e:@@ -176,7 +184,13 @@ def chat_stream(span = tracer.start_span(span_name, kind=SpanKind.CLIENT)return TracedSyncStream(- awaited_wrapped, span, span_name, kwargs, server_address, server_port+ awaited_wrapped,+ span,+ span_name,+ kwargs,+ server_address,+ server_port,+ event_provider=event_provider,)return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/cohere/utils.py b/sdk/python/src/openlit/instrumentation/cohere/utils.pyindex d39925bd..be9e0595 100644--- a/sdk/python/src/openlit/instrumentation/cohere/utils.py+++ b/sdk/python/src/openlit/instrumentation/cohere/utils.py@@ -2,6 +2,8 @@Cohere OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -15,9 +17,12 @@ from openlit.__helpers import (common_span_attributes,record_completion_metrics,record_embedding_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -49,6 +54,176 @@ def format_content(messages):return "\n".join(formatted_messages)+def build_input_messages(messages):+ """+ Convert Cohere request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ structured_messages = []+ for msg in messages or []:+ if isinstance(msg, dict):+ role = msg.get("role", "user")+ content = msg.get("content", "")+ else:+ role = getattr(msg, "role", "user")+ content = getattr(msg, "content", "")+ parts = []+ if isinstance(content, str):+ parts.append({"type": "text", "content": content})+ elif isinstance(content, list):+ for part in content:+ if isinstance(part, dict):+ if part.get("type") == "text":+ parts.append({"type": "text", "content": part.get("text", "")})+ else:+ parts.append({"type": "text", "content": str(content)})+ if parts:+ structured_messages.append({"role": role, "parts": parts})+ return structured_messages+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Cohere response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ if response_text:+ parts.append({"type": "text", "content": response_text})+ if tool_calls:+ for tool_call in tool_calls:+ tc = tool_call if isinstance(tool_call, dict) else {}+ fn = tc.get("function", {}) or {}+ parts.append(+ {+ "type": "tool_call",+ "id": tc.get("id", ""),+ "name": fn.get("name", ""),+ "arguments": fn.get("arguments", ""),+ }+ )+ finish_reason_map = {+ "complete": "stop",+ "stop": "stop",+ "max_tokens": "max_tokens",+ "tool_use": "tool_calls",+ }+ otel_finish = finish_reason_map.get(finish_reason, finish_reason or "stop")+ return [{"role": "assistant", "parts": parts, "finish_reason": otel_finish}]+++def build_tool_definitions(tools):+ """Extract tool definitions from Cohere request. Returns tools or None."""+ return tools if tools else None+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type" and value is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "top_k" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_K] = value+ elif key == "frequency_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n") and value is not None and value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value if value is not None else 0+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value if value is not None else 0+ elif key == "error_type" and value is not None:+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions" and value is not None:+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "conversation_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def process_chunk(scope, chunk):"""Process a chunk of response data and update state.@@ -124,9 +299,15 @@ def process_chunk(scope, chunk):if chunked.get("type") == "message-end":delta = chunked.get("delta", {})scope._finish_reason = delta.get("finish_reason", "")- usage = delta.get("usage", {}).get("billed_units", {})- scope._input_tokens = usage.get("input_tokens", 0)- scope._output_tokens = usage.get("output_tokens", 0)+ usage = delta.get("usage", {})+ billed = usage.get("billed_units", {})+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = billed.get("input_tokens", 0)+ scope._output_tokens = billed.get("output_tokens", 0)+ scope._cache_read_input_tokens = getattr(scope, "_cache_read_input_tokens", 0)+ scope._cache_creation_input_tokens = getattr(+ scope, "_cache_creation_input_tokens", 0+ )scope._end_time = time.time()@@ -140,15 +321,16 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process chat request and generate Telemetry"""+ scope._end_time = getattr(scope, "_end_time", None) or time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "command-r-plus-08-2024")cost = get_chat_model_cost(@@ -213,20 +395,7 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)- # Span Attributes for Cost and Tokens- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,- )- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Span Attributes for Tools - optimized+ # Span Attributes for Toolsif scope._tools:tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]@@ -262,30 +431,84 @@ def common_chat_logic(SemanticConvention.GEN_AI_CONTENT_REASONING, scope._tool_plan)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ scope._input_tokens + scope._output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)-- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },++ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, scope._tools)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ # Emit inference event+ if event_provider:+ try:+ tool_defs = build_tool_definitions(scope._kwargs.get("tools"))+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("p"),+ "top_k": scope._kwargs.get("k"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop_sequences"),+ "seed": scope._kwargs.get("seed"),+ "input_tokens": scope._input_tokens,+ "output_tokens": scope._output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=scope._response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=tool_defs,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_completion_metrics(metrics,@@ -316,6 +539,7 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process streaming chat request and generate Telemetry@@ -331,6 +555,7 @@ def process_streaming_chat_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -348,6 +573,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -370,12 +596,13 @@ def process_chat_response(scope._llmresponse = ""scope._response_id = response_dict.get("id")scope._response_model = request_model- scope._input_tokens = (- response_dict.get("usage", {}).get("billed_units", {}).get("input_tokens", 0)- )- scope._output_tokens = (- response_dict.get("usage", {}).get("billed_units", {}).get("output_tokens", 0)- )+ usage = response_dict.get("usage", {})+ billed = usage.get("billed_units", {})+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = billed.get("input_tokens", 0)+ scope._output_tokens = billed.get("output_tokens", 0)+ scope._cache_read_input_tokens = 0 # Cohere does not expose cached tokens in API+ scope._cache_creation_input_tokens = 0scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_port@@ -401,6 +628,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/elevenlabs/__init__.py b/sdk/python/src/openlit/instrumentation/elevenlabs/__init__.pyindex 09fbbb6a..3deda45b 100644--- a/sdk/python/src/openlit/instrumentation/elevenlabs/__init__.py+++ b/sdk/python/src/openlit/instrumentation/elevenlabs/__init__.py@@ -27,6 +27,7 @@ class ElevenLabsInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("elevenlabs")# sync text_to_speech.convert@@ -43,6 +44,7 @@ class ElevenLabsInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -60,6 +62,7 @@ class ElevenLabsInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/elevenlabs/async_elevenlabs.py b/sdk/python/src/openlit/instrumentation/elevenlabs/async_elevenlabs.pyindex f392dcb7..73b7ce02 100644--- a/sdk/python/src/openlit/instrumentation/elevenlabs/async_elevenlabs.py+++ b/sdk/python/src/openlit/instrumentation/elevenlabs/async_elevenlabs.py@@ -19,6 +19,7 @@ def async_generate(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -57,6 +58,7 @@ def async_generate(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:diff --git a/sdk/python/src/openlit/instrumentation/elevenlabs/elevenlabs.py b/sdk/python/src/openlit/instrumentation/elevenlabs/elevenlabs.pyindex 1a7de5b6..653b14aa 100644--- a/sdk/python/src/openlit/instrumentation/elevenlabs/elevenlabs.py+++ b/sdk/python/src/openlit/instrumentation/elevenlabs/elevenlabs.py@@ -19,6 +19,7 @@ def generate(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -57,6 +58,7 @@ def generate(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:diff --git a/sdk/python/src/openlit/instrumentation/elevenlabs/utils.py b/sdk/python/src/openlit/instrumentation/elevenlabs/utils.pyindex 721de560..5d7f5bdf 100644--- a/sdk/python/src/openlit/instrumentation/elevenlabs/utils.py+++ b/sdk/python/src/openlit/instrumentation/elevenlabs/utils.py@@ -2,21 +2,22 @@ElevenLabs OpenTelemetry instrumentation utility functions"""+import json+import loggingimport time-from opentelemetry.sdk.resources import (- SERVICE_NAME,- TELEMETRY_SDK_NAME,- DEPLOYMENT_ENVIRONMENT,-)from opentelemetry.trace import Status, StatusCodefrom openlit.__helpers import (+ common_span_attributes,get_audio_model_cost,- create_metrics_attributes,+ record_audio_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(text):"""@@ -25,75 +26,142 @@ def format_content(text):return str(text) if text else ""-def common_span_attributes(- scope,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,- request_model,- response_model,- environment,- application_name,- is_stream,- tbt,- ttft,- version,-):+def build_input_messages(text):"""- Set common span attributes for both chat and RAG operations.+ Convert ElevenLabs TTS request text to OTel input message structure.+ Follows gen-ai-input-messages schema."""+ if not text:+ return []+ return [{"role": "user", "parts": [{"type": "text", "content": str(text)}]}]- scope._span.set_attribute(TELEMETRY_SDK_NAME, "openlit")- scope._span.set_attribute(SemanticConvention.GEN_AI_OPERATION, gen_ai_operation)- scope._span.set_attribute(- SemanticConvention.GEN_AI_PROVIDER_NAME, GEN_AI_PROVIDER_NAME- )- scope._span.set_attribute(SemanticConvention.SERVER_ADDRESS, server_address)- scope._span.set_attribute(SemanticConvention.SERVER_PORT, server_port)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, request_model)- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_MODEL, scope._response_model- )- scope._span.set_attribute(DEPLOYMENT_ENVIRONMENT, environment)- scope._span.set_attribute(SERVICE_NAME, application_name)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, is_stream)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TBT, scope._tbt)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TTFT, scope._ttft)- scope._span.set_attribute(SemanticConvention.GEN_AI_SDK_VERSION, version)+def build_output_messages(output_type="speech"):+ """+ Convert ElevenLabs TTS response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ return [+ {+ "role": "assistant",+ "parts": [{"type": "text", "content": "[audio generated]"}],+ "finish_reason": "stop",+ }+ ]-def record_audio_metrics(- metrics,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,request_model,response_model,- environment,- application_name,- start_time,- end_time,- cost,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,):"""- Record audio generation metrics for the operation.+ Emit gen_ai.client.inference.operation.details event."""+ try:+ if not event_provider:+ return- attributes = create_metrics_attributes(- operation=gen_ai_operation,- system=GEN_AI_PROVIDER_NAME,- server_address=server_address,- server_port=server_port,- request_model=request_model,- response_model=response_model,- service_name=application_name,- deployment_environment=environment,- )- metrics["genai_client_operation_duration"].record(end_time - start_time, attributes)- metrics["genai_requests"].add(1, attributes)- metrics["genai_cost"].record(cost, attributes)+ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions++ for key, value in extra_attrs.items():+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value++ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)++ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)def common_audio_logic(@@ -106,16 +174,17 @@ def common_audio_logic(capture_message_content,disable_metrics,version,+ event_provider=None,):"""Process audio generation request and generate Telemetry"""-text = format_content(scope._kwargs.get("text", ""))request_model = scope._kwargs.get("model", scope._kwargs.get("model_id", "eleven_multilingual_v2"))- is_stream = False # ElevenLabs audio generation is not streaming+ response_model = getattr(scope, "_response_model", request_model)+ is_stream = Falsecost = get_audio_model_cost(request_model, pricing_info, text)@@ -127,7 +196,7 @@ def common_audio_logic(scope._server_address,scope._server_port,request_model,- request_model,+ response_model,environment,application_name,is_stream,@@ -136,8 +205,15 @@ def common_audio_logic(version,)- # Span Attributes for Cost and Tokens- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)+ # Span Attributes for Request parameters+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_AUDIO_VOICE,+ scope._kwargs.get("voice_id", ""),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_AUDIO_SETTINGS,+ str(scope._kwargs.get("voice_settings", "")),+ )# Span Attributes for Response parametersscope._span.set_attribute(@@ -145,26 +221,68 @@ def common_audio_logic(scope._kwargs.get("output_format", "mp3_44100_128"),)- # Audio-specific span attributes+ # Span Attributes for Cost and Tokens+ input_tokens = getattr(scope, "_input_tokens", 0)+ output_tokens = getattr(scope, "_output_tokens", 0)scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_AUDIO_VOICE, scope._kwargs.get("voice_id", "")+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_AUDIO_SETTINGS,- str(scope._kwargs.get("voice_settings", "")),+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )# Span Attributes for Contentif capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, text)-- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: text,- },+ input_msgs = build_input_messages(scope._kwargs.get("text", ""))+ output_msgs = build_output_messages(+ scope._kwargs.get("output_format", "mp3_44100_128"))+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ output_type = scope._kwargs.get("output_format", "mp3_44100_128")+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "output_type": output_type,+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_AUDIO,+ request_model=request_model,+ response_model=response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -177,7 +295,7 @@ def common_audio_logic(scope._server_address,scope._server_port,request_model,- request_model,+ response_model,environment,application_name,scope._start_time,@@ -202,6 +320,7 @@ def process_audio_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,):"""Process audio generation request and generate Telemetry@@ -216,13 +335,18 @@ def process_audio_response(scope._kwargs = kwargsscope._args = args- # Initialize streaming and timing values for ElevenLabs audio generationscope._response_model = kwargs.get("model", kwargs.get("model_id", "eleven_multilingual_v2"))scope._tbt = 0.0scope._ttft = scope._end_time - scope._start_time+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = 0+ scope._output_tokens = 0+ scope._cache_read_input_tokens = 0+ scope._cache_creation_input_tokens = 0+common_audio_logic(scope,gen_ai_endpoint,@@ -233,6 +357,7 @@ def process_audio_response(capture_message_content,disable_metrics,version,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.py b/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.pyindex f9f9a1b4..aa16ea2d 100644--- a/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.py+++ b/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.py@@ -110,8 +110,12 @@ def async_generate_stream(self._span_name = span_nameself._llmresponse = ""self._finish_reason = ""- self._output_tokens = ""- self._input_tokens = ""+ self._response_id = ""+ self._input_tokens = 0+ self._output_tokens = 0+ self._reasoning_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._response_model = ""self._tools = Nonediff --git a/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.py b/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.pyindex 694cd4d4..4046cd5e 100644--- a/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.py+++ b/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.py@@ -110,8 +110,12 @@ def generate_stream(self._span_name = span_nameself._llmresponse = ""self._finish_reason = ""- self._output_tokens = ""- self._input_tokens = ""+ self._response_id = ""+ self._input_tokens = 0+ self._output_tokens = 0+ self._reasoning_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._response_model = ""self._tools = Nonediff --git a/sdk/python/src/openlit/instrumentation/google_ai_studio/utils.py b/sdk/python/src/openlit/instrumentation/google_ai_studio/utils.pyindex ccdf9665..14da6648 100644--- a/sdk/python/src/openlit/instrumentation/google_ai_studio/utils.py+++ b/sdk/python/src/openlit/instrumentation/google_ai_studio/utils.py@@ -2,22 +2,26 @@Google AI Studio OpenTelemetry instrumentation utility functions"""+import json+import loggingimport time-from opentelemetry.sdk.resources import (- SERVICE_NAME,- TELEMETRY_SDK_NAME,- DEPLOYMENT_ENVIRONMENT,-)+from opentelemetry.trace import Status, StatusCode+from openlit.__helpers import (calculate_ttft,response_as_dict,calculate_tbt,get_chat_model_cost,- create_metrics_attributes,+ common_span_attributes,+ general_tokens,+ otel_event,+ record_completion_metrics,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -75,6 +79,7 @@ def format_content(messages):def build_input_messages(contents, system_instruction=None):"""Convert Google AI Studio contents to OTel input message structure.+ Follows gen-ai-input-messages schema.Args:contents: Contents array from Google AI Studio request@@ -83,9 +88,6 @@ def build_input_messages(contents, system_instruction=None):Returns:List of ChatMessage objects with role and parts"""- import logging-- logger = logging.getLogger(__name__)if not contents:return []@@ -176,6 +178,7 @@ def build_input_messages(contents, system_instruction=None):def build_output_messages(response_text, finish_reason, function_calls=None):"""Convert Google AI Studio response to OTel output message structure.+ Follows gen-ai-output-messages schema.Args:response_text: Response text from model@@ -185,9 +188,6 @@ def build_output_messages(response_text, finish_reason, function_calls=None):Returns:List with single OutputMessage"""- import logging-- logger = logging.getLogger(__name__)parts = []@@ -240,10 +240,6 @@ def build_tool_definitions(tools):Returns:List of tool definition objects or None"""- import logging-- logger = logging.getLogger(__name__)-if not tools:return None@@ -301,6 +297,27 @@ def build_tool_definitions(tools):return None+def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -328,16 +345,10 @@ def emit_inference_event(server_port: Server port**extra_attrs: Additional attributes (temperature, max_tokens, etc.)"""- import logging-- logger = logging.getLogger(__name__)-try:if not event_provider:return- from openlit.__helpers import otel_event-# Build event attributesattributes = {SemanticConvention.GEN_AI_OPERATION: operation_name,@@ -395,6 +406,16 @@ def emit_inference_event(attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ else:+ attributes[key] = value# Create and emit eventevent = otel_event(@@ -409,45 +430,57 @@ def emit_inference_event(logger.warning("Failed to emit inference event: %s", e, exc_info=True)-def process_chunk(self, chunk):+def process_chunk(scope, chunk):"""Process a chunk of response data and update state."""end_time = time.time()- # Record the timestamp for the current chunk- self._timestamps.append(end_time)+ scope._timestamps.append(end_time)- if len(self._timestamps) == 1:- # Calculate time to first chunk- self._ttft = calculate_ttft(self._timestamps, self._start_time)+ if len(scope._timestamps) == 1:+ scope._ttft = calculate_ttft(scope._timestamps, scope._start_time)chunked = response_as_dict(chunk)- self._response_id = str(chunked.get("response_id"))+ scope._response_id = str(chunked.get("response_id", ""))- # Extract usage including cache tokens (Google prompt caching)- usage_metadata = chunked.get("usage_metadata", {})- self._input_tokens = usage_metadata.get("prompt_token_count", 0)- self._output_tokens = usage_metadata.get("candidates_token_count", 0)- self._reasoning_tokens = usage_metadata.get("thoughts_token_count") or 0- self._cache_read_input_tokens = usage_metadata.get("cached_content_token_count", 0)+ # Handle token usage including reasoning tokens and cached tokens+ usage_metadata = chunked.get("usage_metadata") or {}+ scope._input_tokens = usage_metadata.get("prompt_token_count", 0) or 0+ scope._output_tokens = usage_metadata.get("candidates_token_count", 0) or 0+ scope._reasoning_tokens = usage_metadata.get("thoughts_token_count", 0) or 0+ scope._cache_read_input_tokens = (+ usage_metadata.get("cached_content_token_count", 0) or 0+ )+ scope._cache_creation_input_tokens = (+ usage_metadata.get("cache_creation_input_tokens", 0) or 0+ )- self._response_model = chunked.get("model_version")+ scope._response_model = chunked.get("model_version")- if chunk.text:- self._llmresponse += str(chunk.text)- self._finish_reason = str(chunked.get("candidates")[0].get("finish_reason"))+ if getattr(chunk, "text", None):+ scope._llmresponse += str(chunk.text)+ candidates = chunked.get("candidates") or []+ if candidates:+ scope._finish_reason = str(candidates[0].get("finish_reason", ""))try:- self._tools = (- chunked.get("candidates", [])[0]- .get("content", {})- .get("parts", [])[0]- .get("function_call", "")- )- except:- self._tools = None+ c0 = (chunked.get("candidates") or [{}])[0]+ parts = (c0.get("content") or {}).get("parts") or []+ fc = parts[0].get("function_call") if parts else None+ scope._tools = fc+ except (IndexError, KeyError, TypeError):+ scope._tools = None+++def _get_config_value(config, key):+ """Get value from config (dict or object)."""+ if config is None:+ return None+ if isinstance(config, dict):+ return config.get(key)+ return getattr(config, key, None)def common_chat_logic(@@ -473,26 +506,37 @@ def common_chat_logic(prompt = format_content(scope._kwargs.get("contents", ""))request_model = scope._kwargs.get("model", "gemini-2.0-flash")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ # Calculate tokens and cost+ if hasattr(scope, "_input_tokens") and scope._input_tokens is not None:+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)- # Set Span attributes (OTel Semconv)- scope._span.set_attribute(TELEMETRY_SDK_NAME, "openlit")- scope._span.set_attribute(- SemanticConvention.GEN_AI_OPERATION,+ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)++ # Common Span Attributes+ common_span_attributes(+ scope,SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_GEMINI,+ scope._server_address,+ scope._server_port,+ request_model,+ scope._response_model,+ environment,+ application_name,+ is_stream,+ scope._tbt,+ scope._ttft,+ version,)- scope._span.set_attribute(- SemanticConvention.GEN_AI_PROVIDER_NAME, SemanticConvention.GEN_AI_SYSTEM_GEMINI- )- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, request_model)- scope._span.set_attribute(SemanticConvention.SERVER_PORT, scope._server_port)- scope._span.set_attribute(SemanticConvention.SERVER_ADDRESS, scope._server_address)- inference_config = scope._kwargs.get("config", {})+ inference_config = scope._kwargs.get("config") or {}- attributes = [+ # Span Attributes for Request parameters+ req_attrs = [(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, "frequency_penalty"),(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, "max_tokens"),(SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY, "presence_penalty"),@@ -501,86 +545,141 @@ def common_chat_logic((SemanticConvention.GEN_AI_REQUEST_TOP_P, "top_p"),(SemanticConvention.GEN_AI_REQUEST_TOP_K, "top_k"),]-- # Set each attribute if the corresponding value exists and is not None- for attribute, key in attributes:- # Use getattr to get the attribute value from the object- value = getattr(inference_config, key, None)+ for attribute, key in req_attrs:+ value = _get_config_value(inference_config, key)if value is not None:scope._span.set_attribute(attribute, value)+ # Span Attributes for Response parameters+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason])scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_MODEL, scope._response_model- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ SemanticConvention.GEN_AI_OUTPUT_TYPE,+ "text" if isinstance(scope._llmresponse, str) else "json",)++ # Span Attributes for Tools+ if hasattr(scope, "_tools") and scope._tools:+ tools = scope._tools if isinstance(scope._tools, dict) else {}+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, tools.get("name", "")+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, str(tools.get("id", ""))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS, str(tools.get("args", ""))+ )++ # Span Attributes for Cost and Tokensscope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_REASONING_TOKENS, scope._reasoning_tokens+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)-scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_TYPE,- "text" if isinstance(scope._llmresponse, str) else "json",+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens)-- scope._span.set_attribute(DEPLOYMENT_ENVIRONMENT, environment)- scope._span.set_attribute(SERVICE_NAME, application_name)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, is_stream)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TBT, scope._tbt)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TTFT, scope._ttft)- scope._span.set_attribute(SemanticConvention.GEN_AI_SDK_VERSION, version)- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens + scope._reasoning_tokens,- )- if scope._tools:+ # Reasoning tokens+ if (+ hasattr(scope, "_reasoning_tokens")+ and scope._reasoning_tokens+ and scope._reasoning_tokens > 0+ ):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("name", "")+ "gen_ai.usage.reasoning_tokens", scope._reasoning_tokens)++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS, str(scope._tools.get("args", ""))+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # Span Attributes for Content+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ system_instruction = _get_config_value(inference_config, "system_instruction")+ input_msgs = build_input_messages(+ scope._kwargs.get("contents", []),+ system_instruction=system_instruction,+ )+ output_msgs = build_output_messages(+ scope._llmresponse,+ scope._finish_reason,+ function_calls=scope._tools if hasattr(scope, "_tools") else None,)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ # gen_ai.system_instructions (Google: from config.system_instruction)+ if system_instruction:+ system_text = ""+ if isinstance(system_instruction, str):+ system_text = system_instruction+ elif hasattr(system_instruction, "parts"):+ for part in system_instruction.parts:+ if hasattr(part, "text") and part.text:+ system_text += part.text+ if system_text:+ system_instr = [{"type": "text", "content": system_text}]+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS,+ json.dumps(system_instr),+ )# Emit inference eventif event_provider:- import logging-- logger = logging.getLogger(__name__)try:- input_msgs = build_input_messages(- scope._kwargs.get("contents", []),- system_instruction=getattr(- inference_config, "system_instruction", None- ),- )- output_msgs = build_output_messages(- scope._llmresponse,- scope._finish_reason,- function_calls=scope._tools,- )tool_defs = build_tool_definitions(- getattr(inference_config, "tools", None)+ _get_config_value(inference_config, "tools"))-+ output_type = "text" if isinstance(scope._llmresponse, str) else "json"+ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": output_type,+ "temperature": _get_config_value(inference_config, "temperature"),+ "max_tokens": _get_config_value(inference_config, "max_tokens"),+ "top_p": _get_config_value(inference_config, "top_p"),+ "top_k": _get_config_value(inference_config, "top_k"),+ "stop_sequences": _get_config_value(+ inference_config, "stop_sequences"+ ),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ if hasattr(scope, "_reasoning_tokens") and scope._reasoning_tokens:+ extra["reasoning_tokens"] = scope._reasoning_tokens+ if system_instruction:+ system_text = ""+ if isinstance(system_instruction, str):+ system_text = system_instruction+ elif hasattr(system_instruction, "parts"):+ for part in system_instruction.parts:+ if hasattr(part, "text") and part.text:+ system_text += part.text+ if system_text:+ extra["system_instructions"] = [+ {"type": "text", "content": system_text}+ ]emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,@@ -591,57 +690,37 @@ def common_chat_logic(tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- finish_reasons=[scope._finish_reason],- temperature=getattr(inference_config, "temperature", None),- max_output_tokens=getattr(inference_config, "max_tokens", None),- top_p=getattr(inference_config, "top_p", None),- top_k=getattr(inference_config, "top_k", None),- stop_sequences=getattr(inference_config, "stop_sequences", None),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,- reasoning_tokens=scope._reasoning_tokens,- cache_read_input_tokens=scope._cache_read_input_tokens- if hasattr(scope, "_cache_read_input_tokens")- and scope._cache_read_input_tokens > 0- else None,+ **extra,)except Exception as e:logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))+ # Record metricsif not disable_metrics:- metrics_attributes = create_metrics_attributes(- service_name=application_name,- deployment_environment=environment,- operation=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- system=SemanticConvention.GEN_AI_SYSTEM_GEMINI,- request_model=request_model,- server_address=scope._server_address,- server_port=scope._server_port,- response_model=scope._response_model,- )-- metrics["genai_client_operation_duration"].record(- scope._end_time - scope._start_time, metrics_attributes- )- metrics["genai_server_tbt"].record(scope._tbt, metrics_attributes)- metrics["genai_server_ttft"].record(scope._ttft, metrics_attributes)- metrics["genai_requests"].add(1, metrics_attributes)- metrics["genai_completion_tokens"].add(scope._output_tokens, metrics_attributes)- metrics["genai_prompt_tokens"].add(scope._input_tokens, metrics_attributes)- metrics["genai_reasoning_tokens"].add(- scope._reasoning_tokens, metrics_attributes- )- metrics["genai_cost"].record(cost, metrics_attributes)- metrics["genai_client_usage_tokens"].record(- scope._input_tokens + scope._output_tokens + scope._reasoning_tokens,- metrics_attributes,+ record_completion_metrics(+ metrics,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_GEMINI,+ scope._server_address,+ scope._server_port,+ request_model,+ scope._response_model,+ environment,+ application_name,+ scope._start_time,+ scope._end_time,+ input_tokens,+ output_tokens,+ cost,+ scope._tbt,+ scope._ttft,)def process_streaming_chat_response(- self,+ scope,pricing_info,environment,application_name,@@ -652,11 +731,11 @@ def process_streaming_chat_response(event_provider=None,):"""- Process chat request and generate Telemetry+ Process streaming chat response and generate telemetry."""common_chat_logic(- self,+ scope,pricing_info,environment,application_name,@@ -700,28 +779,34 @@ def process_chat_response(self._span = spanself._llmresponse = str(response.text)- # Extract usage including cache tokens (Google prompt caching)- usage_metadata = response_dict.get("usage_metadata", {})- self._input_tokens = usage_metadata.get("prompt_token_count", 0)- self._output_tokens = usage_metadata.get("candidates_token_count", 0)- self._reasoning_tokens = usage_metadata.get("thoughts_token_count") or 0- self._cache_read_input_tokens = usage_metadata.get("cached_content_token_count", 0)+ # Handle token usage including reasoning tokens and cached tokens+ usage_metadata = response_dict.get("usage_metadata") or {}+ self._input_tokens = usage_metadata.get("prompt_token_count", 0) or 0+ self._output_tokens = usage_metadata.get("candidates_token_count", 0) or 0+ self._reasoning_tokens = usage_metadata.get("thoughts_token_count", 0) or 0+ self._cache_read_input_tokens = (+ usage_metadata.get("cached_content_token_count", 0) or 0+ )+ self._cache_creation_input_tokens = (+ usage_metadata.get("cache_creation_input_tokens", 0) or 0+ )+ self._response_id = str(response_dict.get("response_id", ""))self._response_model = response_dict.get("model_version")self._timestamps = []self._ttft, self._tbt = self._end_time - self._start_time, 0self._server_address, self._server_port = server_address, server_portself._kwargs = kwargs- self._finish_reason = str(response_dict.get("candidates")[0].get("finish_reason"))+ candidates = response_dict.get("candidates") or []+ self._finish_reason = (+ str(candidates[0].get("finish_reason", "")) if candidates else ""+ )try:- self._tools = (- response_dict.get("candidates", [])[0]- .get("content", {})- .get("parts", [])[0]- .get("function_call", "")- )- except:+ c0 = (response_dict.get("candidates") or [{}])[0]+ parts = (c0.get("content") or {}).get("parts") or []+ self._tools = parts[0].get("function_call") if parts else None+ except (IndexError, KeyError, TypeError):self._tools = Nonecommon_chat_logic(diff --git a/sdk/python/src/openlit/instrumentation/gpt4all/__init__.py b/sdk/python/src/openlit/instrumentation/gpt4all/__init__.pyindex 0a55334f..e1dc05be 100644--- a/sdk/python/src/openlit/instrumentation/gpt4all/__init__.py+++ b/sdk/python/src/openlit/instrumentation/gpt4all/__init__.py@@ -26,6 +26,7 @@ class GPT4AllInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("gpt4all")# generate@@ -41,6 +42,7 @@ class GPT4AllInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider=event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/gpt4all/gpt4all.py b/sdk/python/src/openlit/instrumentation/gpt4all/gpt4all.pyindex 3ebef655..956e5ab6 100644--- a/sdk/python/src/openlit/instrumentation/gpt4all/gpt4all.py+++ b/sdk/python/src/openlit/instrumentation/gpt4all/gpt4all.py@@ -23,6 +23,7 @@ def generate(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -43,6 +44,7 @@ def generate(server_address,server_port,request_model,+ event_provider=None,):self.__wrapped__ = wrappedself._span = span@@ -59,6 +61,9 @@ def generate(self._server_address = server_addressself._server_port = server_portself._tools = None+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0+ self._event_provider = event_providerdef __enter__(self):self.__wrapped__.__enter__()@@ -93,6 +98,7 @@ def generate(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=self._event_provider,)except Exception as e:@@ -131,6 +137,7 @@ def generate(server_address,server_port,request_model,+ event_provider=event_provider,)# Handling for non-streaming responses@@ -156,6 +163,7 @@ def generate(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:diff --git a/sdk/python/src/openlit/instrumentation/gpt4all/utils.py b/sdk/python/src/openlit/instrumentation/gpt4all/utils.pyindex b7f419cf..f57e13f4 100644--- a/sdk/python/src/openlit/instrumentation/gpt4all/utils.py+++ b/sdk/python/src/openlit/instrumentation/gpt4all/utils.py@@ -2,25 +2,27 @@GPT4All OpenTelemetry instrumentation utility functions"""+import json+import loggingimport time-from opentelemetry.sdk.resources import (- SERVICE_NAME,- TELEMETRY_SDK_NAME,- DEPLOYMENT_ENVIRONMENT,-)from opentelemetry.trace import Status, StatusCodefrom openlit.__helpers import (calculate_ttft,calculate_tbt,general_tokens,- create_metrics_attributes,get_chat_model_cost,get_embed_model_cost,+ common_span_attributes,+ record_completion_metrics,+ record_embedding_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(prompt):"""@@ -29,142 +31,165 @@ def format_content(prompt):return str(prompt) if prompt else ""-def process_chunk(scope, chunk):+def build_input_messages(prompt):"""- Process a chunk of response data and update state.+ Convert GPT4All request prompt to OTel input message structure.+ Follows gen-ai-input-messages schema."""+ content = str(prompt) if prompt else ""+ return [{"role": "user", "parts": [{"type": "text", "content": content}]}]- end_time = time.time()- # Record the timestamp for the current chunk- scope._timestamps.append(end_time)-- if len(scope._timestamps) == 1:- # Calculate time to first chunk- scope._ttft = calculate_ttft(scope._timestamps, scope._start_time)- scope._llmresponse += chunk- scope._end_time = time.time()---def common_span_attributes(- scope,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,- request_model,- response_model,- environment,- application_name,- is_stream,- tbt,- ttft,- version,-):+def build_output_messages(response_text, finish_reason, tool_calls=None):"""- Set common span attributes for both generate and embed operations.+ Convert GPT4All response to OTel output message structure.+ Follows gen-ai-output-messages schema."""-- scope._span.set_attribute(TELEMETRY_SDK_NAME, "openlit")- scope._span.set_attribute(SemanticConvention.GEN_AI_OPERATION, gen_ai_operation)- scope._span.set_attribute(- SemanticConvention.GEN_AI_PROVIDER_NAME, GEN_AI_PROVIDER_NAME- )- scope._span.set_attribute(SemanticConvention.SERVER_ADDRESS, server_address)- scope._span.set_attribute(SemanticConvention.SERVER_PORT, server_port)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, request_model)- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, response_model)- scope._span.set_attribute(DEPLOYMENT_ENVIRONMENT, environment)- scope._span.set_attribute(SERVICE_NAME, application_name)- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, is_stream)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt)- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft)- scope._span.set_attribute(SemanticConvention.GEN_AI_SDK_VERSION, version)---def record_completion_metrics(- metrics,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,+ parts = []+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls:+ tools = tool_calls if isinstance(tool_calls, list) else [tool_calls]+ for t in tools:+ tc = t if isinstance(t, dict) else {}+ fn = tc.get("function", {}) or {}+ parts.append(+ {+ "type": "tool_call",+ "id": tc.get("id", ""),+ "name": fn.get("name", ""),+ "arguments": fn.get("arguments", ""),+ }+ )+ reason = finish_reason if finish_reason else "stop"+ return [{"role": "assistant", "parts": parts, "finish_reason": reason}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,request_model,response_model,- environment,- application_name,- start_time,- end_time,- input_tokens,- output_tokens,- cost,- tbt=None,- ttft=None,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,):"""- Record completion-specific metrics for the operation.+ Emit gen_ai.client.inference.operation.details event."""-- attributes = create_metrics_attributes(- operation=gen_ai_operation,- system=GEN_AI_PROVIDER_NAME,- server_address=server_address,- server_port=server_port,- request_model=request_model,- response_model=response_model,- service_name=application_name,- deployment_environment=environment,- )- metrics["genai_client_operation_duration"].record(end_time - start_time, attributes)- metrics["genai_requests"].add(1, attributes)- metrics["genai_prompt_tokens"].add(input_tokens, attributes)- metrics["genai_completion_tokens"].add(output_tokens, attributes)- metrics["genai_client_usage_tokens"].record(- input_tokens + output_tokens, attributes- )- metrics["genai_cost"].record(cost, attributes)- if tbt is not None:- metrics["genai_server_tbt"].record(tbt, attributes)- if ttft is not None:- metrics["genai_server_ttft"].record(ttft, attributes)+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type" and value is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "top_k" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_K] = value+ elif key == "frequency_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n") and value is not None and value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value if value is not None else 0+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value if value is not None else 0+ elif key == "error_type" and value is not None:+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions" and value is not None:+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "conversation_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)-def record_embedding_metrics(- metrics,- gen_ai_operation,- GEN_AI_PROVIDER_NAME,- server_address,- server_port,- request_model,- response_model,- environment,- application_name,- start_time,- end_time,- input_tokens,- cost,-):+def process_chunk(scope, chunk):"""- Record embedding-specific metrics for the operation.+ Process a chunk of response data and update state."""- attributes = create_metrics_attributes(- operation=gen_ai_operation,- system=GEN_AI_PROVIDER_NAME,- server_address=server_address,- server_port=server_port,- request_model=request_model,- response_model=response_model,- service_name=application_name,- deployment_environment=environment,- )- metrics["genai_client_usage_tokens"].record(input_tokens, attributes)- metrics["genai_client_operation_duration"].record(end_time - start_time, attributes)- metrics["genai_requests"].add(1, attributes)- metrics["genai_prompt_tokens"].add(input_tokens, attributes)- metrics["genai_cost"].record(cost, attributes)+ end_time = time.time()+ # Record the timestamp for the current chunk+ scope._timestamps.append(end_time)+ if len(scope._timestamps) == 1:+ # Calculate time to first chunk+ scope._ttft = calculate_ttft(scope._timestamps, scope._start_time)-def common_t2s_logic(+ scope._llmresponse += chunk+ scope._end_time = time.time()+++def common_generate_logic(scope,pricing_info,environment,@@ -174,6 +199,7 @@ def common_t2s_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process generate request and generate Telemetry@@ -186,10 +212,15 @@ def common_t2s_logic(scope._kwargs.get("prompt") or (scope._args[0] if scope._args else "") or "")request_model = scope._request_model+ response_model = getattr(scope, "_response_model", request_model)- # Calculate tokens using input prompt and aggregated response- input_tokens = general_tokens(prompt)- output_tokens = general_tokens(scope._llmresponse)+ # Calculate tokens from scope (may be set by caller)+ input_tokens = getattr(scope, "_input_tokens", None)+ if input_tokens is None:+ input_tokens = general_tokens(prompt)+ output_tokens = getattr(scope, "_output_tokens", None)+ if output_tokens is None:+ output_tokens = general_tokens(scope._llmresponse)cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)@@ -201,7 +232,7 @@ def common_t2s_logic(scope._server_address,scope._server_port,request_model,- request_model,+ response_model,environment,application_name,is_stream,@@ -232,10 +263,46 @@ def common_t2s_logic(scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TOP_K, scope._kwargs.get("top_k", 40))++ # Span Attributes for Response parametersscope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE,"text" if isinstance(scope._llmresponse, str) else "json",)+ finish_reason = getattr(scope, "_finish_reason", None) or "stop"+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [finish_reason]+ )+ response_id = getattr(scope, "_response_id", None)+ if response_id is not None:+ scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, response_id)++ # Span Attributes for Tools+ if scope._tools:+ tools_list = scope._tools if isinstance(scope._tools, list) else [scope._tools]+ names = []+ ids = []+ args_list = []+ for t in tools_list:+ item = t if isinstance(t, dict) else {}+ fn = item.get("function") or {}+ if isinstance(fn, dict):+ names.append(fn.get("name", ""))+ args_list.append(str(fn.get("arguments", "")))+ else:+ names.append("")+ args_list.append("")+ ids.append(str(item.get("id", "")))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, ",".join(names) if names else ""+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, ",".join(ids) if ids else ""+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS,+ ",".join(args_list) if args_list else "",+ )# Span Attributes for Cost and Tokensscope._span.set_attribute(@@ -249,43 +316,59 @@ def common_t2s_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Tools- if scope._tools:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("function", "")- ).get("name", "")- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS,- str(scope._tools.get("function", "").get("arguments", "")),+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)# Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse- )-- # To be removed one the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },- )- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },+ input_msgs = build_input_messages(prompt)+ output_msgs = build_output_messages(+ scope._llmresponse,+ getattr(scope, "_finish_reason", None),+ getattr(scope, "_tools", None),+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ if capture_message_content and event_provider:+ cache_read = getattr(scope, "_cache_read_input_tokens", 0)+ cache_creation = getattr(scope, "_cache_creation_input_tokens", 0)+ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [finish_reason],+ "output_type": "text" if isinstance(scope._llmresponse, str) else "json",+ "temperature": scope._kwargs.get("temp"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "top_k": scope._kwargs.get("top_k"),+ "frequency_penalty": scope._kwargs.get("repeat_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": cache_read,+ "cache_creation_input_tokens": cache_creation,+ }+ emit_inference_event(+ event_provider,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model,+ response_model,+ input_messages=json.dumps(input_msgs) if input_msgs else None,+ output_messages=json.dumps(output_msgs) if output_msgs else None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_completion_metrics(metrics,@@ -294,7 +377,7 @@ def common_t2s_logic(scope._server_address,scope._server_port,request_model,- request_model,+ response_model,environment,application_name,scope._start_time,@@ -356,18 +439,14 @@ def common_embedding_logic(# Span Attributes for Contentif capture_message_content:- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: str(- scope._kwargs.get("input", "")- ),- },+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ str(scope._kwargs.get("text", "") or scope._kwargs.get("input", "")),)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_embedding_metrics(metrics,@@ -395,11 +474,12 @@ def process_streaming_generate_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process generate request and generate Telemetry"""- common_t2s_logic(+ common_generate_logic(scope,pricing_info,environment,@@ -409,6 +489,7 @@ def process_streaming_generate_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -428,6 +509,7 @@ def process_generate_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,):"""Process generate request and generate Telemetry@@ -447,7 +529,14 @@ def process_generate_response(scope._args = argsscope._tools = None- common_t2s_logic(+ # Handle token usage including reasoning tokens and cached tokens+ prompt = format_content(kwargs.get("prompt") or (args[0] if args else "") or "")+ scope._input_tokens = general_tokens(prompt)+ scope._output_tokens = general_tokens(scope._llmresponse)+ scope._cache_read_input_tokens = 0+ scope._cache_creation_input_tokens = 0++ common_generate_logic(scope,pricing_info,environment,@@ -457,6 +546,7 @@ def process_generate_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/groq/__init__.py b/sdk/python/src/openlit/instrumentation/groq/__init__.pyindex b82afc62..31791c44 100644--- a/sdk/python/src/openlit/instrumentation/groq/__init__.py+++ b/sdk/python/src/openlit/instrumentation/groq/__init__.py@@ -27,6 +27,7 @@ class GroqInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("groq")# Chat completions@@ -42,6 +43,7 @@ class GroqInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -58,6 +60,7 @@ class GroqInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/groq/async_groq.py b/sdk/python/src/openlit/instrumentation/groq/async_groq.pyindex adafb85d..ab836700 100644--- a/sdk/python/src/openlit/instrumentation/groq/async_groq.py+++ b/sdk/python/src/openlit/instrumentation/groq/async_groq.py@@ -22,6 +22,7 @@ def async_chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -53,6 +54,8 @@ def async_chat(self._system_fingerprint = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsself._start_time = time.time()@@ -96,6 +99,7 @@ def async_chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:@@ -144,6 +148,7 @@ def async_chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/groq/groq.py b/sdk/python/src/openlit/instrumentation/groq/groq.pyindex 724fa8e8..43159664 100644--- a/sdk/python/src/openlit/instrumentation/groq/groq.py+++ b/sdk/python/src/openlit/instrumentation/groq/groq.py@@ -22,6 +22,7 @@ def chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -53,6 +54,8 @@ def chat(self._system_fingerprint = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsself._start_time = time.time()@@ -96,6 +99,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:@@ -144,6 +148,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/groq/utils.py b/sdk/python/src/openlit/instrumentation/groq/utils.pyindex 48cbc82c..a7c87bb1 100644--- a/sdk/python/src/openlit/instrumentation/groq/utils.py+++ b/sdk/python/src/openlit/instrumentation/groq/utils.py@@ -2,6 +2,8 @@Groq OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -10,12 +12,16 @@ from openlit.__helpers import (calculate_ttft,response_as_dict,calculate_tbt,+ general_tokens,get_chat_model_cost,common_span_attributes,record_completion_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -41,6 +47,208 @@ def format_content(messages):return "\n".join(formatted_messages)+def build_input_messages(messages):+ """+ Convert Groq request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not messages:+ return []+ otel_messages = []+ for message in messages:+ try:+ role = (+ message.get("role", "user")+ if isinstance(message, dict)+ else getattr(message, "role", "user")+ )+ content = (+ message.get("content", "")+ if isinstance(message, dict)+ else getattr(message, "content", "")+ )+ parts = []+ if isinstance(content, list):+ for item in content:+ if isinstance(item, dict) and item.get("type") == "text":+ text = item.get("text", "")+ if text:+ parts.append({"type": "text", "content": text})+ elif isinstance(item, dict) and item.get("type") == "image_url":+ url = (item.get("image_url") or {}).get("url", "")+ if url and not str(url).startswith("data:"):+ parts.append(+ {"type": "uri", "modality": "image", "uri": url}+ )+ elif isinstance(content, str) and content:+ parts.append({"type": "text", "content": content})+ if role == "tool":+ tool_call_id = (+ message.get("tool_call_id", "")+ if isinstance(message, dict)+ else getattr(message, "tool_call_id", "")+ )+ tool_content = content if isinstance(content, str) else str(content)+ parts = [+ {+ "type": "tool_call_response",+ "id": tool_call_id,+ "response": tool_content,+ }+ ]+ if parts:+ otel_messages.append({"role": role, "parts": parts})+ except Exception as e:+ logger.warning("Failed to process input message: %s", e, exc_info=True)+ continue+ return otel_messages+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Groq response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ try:+ if response_text:+ parts.append({"type": "text", "content": response_text})+ if tool_calls:+ tools = tool_calls if isinstance(tool_calls, list) else [tool_calls]+ for t in tools:+ if isinstance(t, dict):+ parts.append(+ {+ "type": "tool_call",+ "id": t.get("id", ""),+ "name": t.get("function", {}).get("name", ""),+ "arguments": t.get("function", {}).get("arguments", {}),+ }+ )+ finish_reason_map = {+ "stop": "stop",+ "length": "length",+ "content_filter": "content_filter",+ "tool_calls": "tool_call",+ "function_call": "tool_call",+ }+ otel_finish = finish_reason_map.get(finish_reason, finish_reason or "stop")+ return [{"role": "assistant", "parts": parts, "finish_reason": otel_finish}]+ except Exception as e:+ logger.warning("Failed to build output messages: %s", e, exc_info=True)+ return [{"role": "assistant", "parts": [], "finish_reason": "stop"}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port is not None:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def process_chunk(scope, chunk):"""Process a chunk of response data and update state.@@ -68,12 +276,16 @@ def process_chunk(scope, chunk):if chunked.get("x_groq") is not None:if chunked.get("x_groq").get("usage") is not None:- scope._input_tokens = (- chunked.get("x_groq").get("usage").get("prompt_tokens")- )- scope._output_tokens = (- chunked.get("x_groq").get("usage").get("completion_tokens")- )+ # Handle token usage including reasoning tokens and cached tokens+ usage = chunked.get("x_groq").get("usage")+ scope._input_tokens = usage.get("prompt_tokens")+ scope._output_tokens = usage.get("completion_tokens")+ scope._cache_read_input_tokens = (+ usage.get("prompt_tokens_details") or {}+ ).get("cached_tokens", 0) or 0+ scope._cache_creation_input_tokens = (+ usage.get("input_tokens_details") or {}+ ).get("cache_creation_tokens", 0) or 0scope._response_id = chunked.get("x_groq").get("id")scope._response_model = chunked.get("x_groq").get("model")scope._finish_reason = chunked.get("choices", [{}])[0].get("finish_reason")@@ -91,10 +303,12 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process chat request and generate Telemetry"""+ scope._end_time = time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)@@ -102,9 +316,19 @@ def common_chat_logic(prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "mixtral-8x7b-32768")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ if (+ hasattr(scope, "_input_tokens")+ and scope._input_tokens is not None+ and hasattr(scope, "_output_tokens")+ and scope._output_tokens is not None+ ):+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributescommon_span_attributes(@@ -114,7 +338,7 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- scope._response_model,+ getattr(scope, "_response_model", request_model),environment,application_name,is_stream,@@ -158,31 +382,23 @@ def common_chat_logic()# Span Attributes for Response parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason])- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_SYSTEM_FINGERPRINT, scope._system_fingerprint- )+ if hasattr(scope, "_system_fingerprint") and scope._system_fingerprint:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_SYSTEM_FINGERPRINT,+ scope._system_fingerprint,+ )scope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE,"text" if isinstance(scope._llmresponse, str) else "json",)- # Span Attributes for Cost and Tokens- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,- )- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-# Span Attributes for Toolsif scope._tools:scope._span.set_attribute(@@ -197,26 +413,76 @@ def common_chat_logic(str(scope._tools.get("function", {}).get("arguments", "")),)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)-- # To be removed one the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },++ # Span Attributes for Content+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, tool_calls=scope._tools)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_completion_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop"),+ "seed": scope._kwargs.get("seed"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=getattr(scope, "_response_model", request_model),+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -229,13 +495,13 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- scope._response_model,+ getattr(scope, "_response_model", request_model),environment,application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -251,6 +517,7 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process chat request and generate Telemetry@@ -266,6 +533,7 @@ def process_streaming_chat_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -283,6 +551,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -301,9 +570,17 @@ def process_chat_response(for choice in response_dict.get("choices", []))scope._response_id = response_dict.get("id")- scope._response_model = response_dict.get("model")- scope._input_tokens = response_dict.get("usage", {}).get("prompt_tokens", 0)- scope._output_tokens = response_dict.get("usage", {}).get("completion_tokens", 0)+ scope._response_model = response_dict.get("model") or request_model+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage", {})+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (usage.get("input_tokens_details") or {}).get(+ "cache_creation_tokens", 0+ ) or 0scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_port@@ -331,6 +608,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/langchain/__init__.py b/sdk/python/src/openlit/instrumentation/langchain/__init__.pyindex 7a9636b9..06f8ee0d 100644--- a/sdk/python/src/openlit/instrumentation/langchain/__init__.py+++ b/sdk/python/src/openlit/instrumentation/langchain/__init__.py@@ -17,6 +17,11 @@ from opentelemetry.trace import SpanKind, Status, StatusCodefrom opentelemetry.trace import set_span_in_contextfrom wrapt import wrap_function_wrapper+from openlit.instrumentation.langchain.utils import (+ build_input_messages,+ common_chat_logic,+)+# Initialize loggerlogger = logging.getLogger(__name__)@@ -41,6 +46,10 @@ class SpanHolder:"model_parameters","prompt_content","input_messages",+ "cache_read_input_tokens",+ "cache_creation_input_tokens",+ "input_messages_raw",+ "prompts",)def __init__(self, span, token=None, start_time=None):@@ -58,6 +67,10 @@ class SpanHolder:self.model_parameters: Dict[str, Any] = {}self.prompt_content = ""self.input_messages: List[Dict[str, Any]] = []+ self.cache_read_input_tokens = 0+ self.cache_creation_input_tokens = 0+ self.input_messages_raw: Any = None+ self.prompts: List[Any] = []def get_duration(self) -> float:"""Calculate duration from start time to now."""@@ -176,190 +189,6 @@ def _extract_model_from_repr(return None-def build_input_messages_from_langchain(messages):- """- Convert LangChain messages to OTel structured input message format.-- Args:- messages: List of LangChain message lists (typically [[msg1, msg2, ...]])-- Returns:- List of ChatMessage objects with role and parts- """- try:- structured_messages = []-- # LangChain wraps messages in nested lists- for msg_list in messages:- for msg in msg_list:- # Extract role and content from LangChain message- role = getattr(msg, "type", "user")- content = getattr(msg, "content", str(msg))-- # Map LangChain roles to OTel roles- role_mapping = {- "system": "system",- "human": "user",- "ai": "assistant",- "tool": "tool",- "function": "tool",- }- otel_role = role_mapping.get(role, "user")-- # Build structured message with parts- structured_msg = {- "role": otel_role,- "parts": [{"type": "text", "content": str(content)}],- }- structured_messages.append(structured_msg)-- return structured_messages- except Exception as e:- # Return empty list on error to prevent instrumentation breakage- logger.debug("Error building input messages from LangChain: %s", e)- return []---def build_input_messages_from_prompts(prompts):- """- Convert LangChain prompts (strings) to OTel structured input message format.-- Args:- prompts: List of prompt strings-- Returns:- List of ChatMessage objects with role and parts- """- try:- structured_messages = []-- for prompt_list in prompts:- # prompts is typically a list of lists- prompt_text = (- prompt_list if isinstance(prompt_list, str) else str(prompt_list)- )-- structured_msg = {- "role": "user",- "parts": [{"type": "text", "content": prompt_text}],- }- structured_messages.append(structured_msg)-- return structured_messages- except Exception as e:- # Return empty list on error to prevent instrumentation breakage- logger.debug("Error building input messages from prompts: %s", e)- return []---def build_output_messages_langchain(completion_content, finish_reason="stop"):- """- Convert LangChain completion content to OTel structured output message format.-- Args:- completion_content: The completion text content- finish_reason: The finish reason (default: "stop")-- Returns:- List with single OutputMessage containing response- """- try:- if not completion_content:- return []-- output_msg = {- "role": "assistant",- "parts": [{"type": "text", "content": str(completion_content)}],- "finish_reason": finish_reason,- }-- return [output_msg]- except Exception as e:- # Return empty list on error to prevent instrumentation breakage- logger.debug("Error building output messages: %s", e)- return []---def emit_inference_event_langchain(- event_provider,- operation_name,- request_model,- response_model,- input_messages,- output_messages,- server_address,- server_port,- **extra_attrs,-):- """- Emit gen_ai.client.inference.operation.details event for LangChain operations.-- Args:- event_provider: The OTel event provider- operation_name: The operation name (e.g., "chat")- request_model: The request model name- response_model: The response model name- input_messages: Structured input messages- output_messages: Structured output messages- server_address: Server address- server_port: Server port- **extra_attrs: Additional attributes (temperature, tokens, etc.)- """- try:- if not event_provider:- return-- from openlit.__helpers import otel_event- from openlit.semcov import SemanticConvention-- # Build event attributes per OTel spec- attributes = {- SemanticConvention.GEN_AI_OPERATION: operation_name,- SemanticConvention.GEN_AI_REQUEST_MODEL: request_model,- SemanticConvention.SERVER_ADDRESS: server_address,- SemanticConvention.SERVER_PORT: server_port,- }-- # Add response model if different from request- if response_model and response_model != request_model:- attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model-- # Add input/output messages- if input_messages:- attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages- if output_messages:- attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages-- # Add optional attributes- optional_attrs = {- "temperature": SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- "max_tokens": SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,- "top_p": SemanticConvention.GEN_AI_REQUEST_TOP_P,- "top_k": SemanticConvention.GEN_AI_REQUEST_TOP_K,- "frequency_penalty": SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- "presence_penalty": SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,- "input_tokens": SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,- "output_tokens": SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,- "finish_reasons": SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,- }-- for key, attr_name in optional_attrs.items():- if key in extra_attrs and extra_attrs[key] is not None:- attributes[attr_name] = extra_attrs[key]-- # Create and emit event- event = otel_event(- name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,- attributes=attributes,- body="", # Per spec, all data in attributes- )-- event_provider.emit(event)-- except Exception as e:- logger.warning("Failed to emit inference event: %s", e, exc_info=True)--def extract_model_name(serialized: Optional[Dict[str, Any]],kwargs: Optional[Dict[str, Any]] = None,@@ -540,8 +369,6 @@ def _create_callback_handler_class(from openlit.__helpers import (general_tokens,- get_chat_model_cost,- record_completion_metrics,record_framework_metrics,calculate_ttft,calculate_tbt,@@ -653,6 +480,7 @@ def _create_callback_handler_class(def _extract_from_generations(self,+ holder,response,input_tokens: int,output_tokens: int,@@ -675,9 +503,9 @@ def _create_callback_handler_class():completion_content = gen_content- # Extract usage from message+ # Extract usage from message (pass holder for cache tokens)tokens = self._extract_message_usage(- gen, input_tokens, output_tokens+ gen, input_tokens, output_tokens, holder=holder)input_tokens, output_tokens = tokens@@ -735,7 +563,7 @@ def _create_callback_handler_class(return resultdef _extract_message_usage(- self, gen, input_tokens: int, output_tokens: int+ self, gen, input_tokens: int, output_tokens: int, holder: Any = None) -> tuple:"""Extract token usage from message metadata.@@ -743,13 +571,15 @@ def _create_callback_handler_class(- usage_metadata (standard LangChain)- response_metadata.usage (Bedrock-Anthropic)- response_metadata.amazon-bedrock-invocationMetrics (Bedrock-Titan)+ When holder is provided, also sets holder.cache_read_input_tokens and+ holder.cache_creation_input_tokens from usage when present."""if not hasattr(gen, "message") or not gen.message:return input_tokens, output_tokensmsg = gen.message- # Check usage_metadata (standard LangChain, Ollama)+ # Handle token usage including reasoning tokens and cached tokensif hasattr(msg, "usage_metadata") and msg.usage_metadata:usage = msg.usage_metadatainput_tokens = (@@ -762,12 +592,61 @@ def _create_callback_handler_class(or usage.get("completion_tokens")or output_tokens)-- # Check response_metadata (Bedrock, etc.)+ if holder is not None:+ # OpenAI-style: prompt_tokens_details.cached_tokens / input_tokens_details.cache_creation_tokens+ prompt_details = (+ usage.get("prompt_tokens_details")+ or usage.get("input_tokens_details")+ or {}+ )+ cached = prompt_details.get("cached_tokens", 0) or 0+ input_details = usage.get("input_tokens_details") or {}+ creation = input_details.get("cache_creation_tokens", 0) or 0+ # LangChain usage_metadata: input_token_details.cache_read / cache_creation+ langchain_input = usage.get("input_token_details") or {}+ if cached == 0:+ cached = langchain_input.get("cache_read", 0) or 0+ if creation == 0:+ creation = langchain_input.get("cache_creation", 0) or 0+ holder.cache_read_input_tokens = cached+ holder.cache_creation_input_tokens = creation++ # Check response_metadata (Bedrock, OpenAI via LangChain, etc.)if hasattr(msg, "response_metadata") and msg.response_metadata:metadata = msg.response_metadataif isinstance(metadata, dict):- # Bedrock-Anthropic style+ # OpenAI-style (e.g. LangChain OpenAI): token_usage with prompt_tokens_details+ token_usage = metadata.get("token_usage")+ if token_usage:+ input_tokens = (+ token_usage.get("prompt_tokens")+ or token_usage.get("input_tokens")+ or input_tokens+ )+ output_tokens = (+ token_usage.get("completion_tokens")+ or token_usage.get("output_tokens")+ or output_tokens+ )+ if holder is not None:+ prompt_details = (+ token_usage.get("prompt_tokens_details")+ or token_usage.get("input_tokens_details")+ or {}+ )+ if holder.cache_read_input_tokens == 0:+ holder.cache_read_input_tokens = (+ prompt_details.get("cached_tokens", 0) or 0+ )+ input_details = (+ token_usage.get("input_tokens_details") or {}+ )+ if holder.cache_creation_input_tokens == 0:+ holder.cache_creation_input_tokens = (+ input_details.get("cache_creation_tokens", 0) or 0+ )++ # Bedrock-Anthropic style: usage with inputTokens/outputTokensif "usage" in metadata:usage = metadata["usage"]input_tokens = usage.get(@@ -975,6 +854,7 @@ def _create_callback_handler_class(# Store model info for laterself.spans[run_id].model_name = model_nameself.spans[run_id].model_parameters = model_params+ self.spans[run_id].prompts = prompts if prompts else []if self._capture_message_content and prompts:prompt_str = "\n".join(prompts)[:5000]@@ -985,7 +865,7 @@ def _create_callback_handler_class(# Store structured prompts for event emissiontry:- structured_prompts = build_input_messages_from_prompts(prompts)+ structured_prompts = build_input_messages(prompts)self.spans[run_id].input_messages = structured_promptsexcept Exception as prompt_err:logger.debug(@@ -1038,9 +918,10 @@ def _create_callback_handler_class(formatted.append(f"{role}: {content}")prompt_str = "\n".join(formatted)[:5000]- # Store prompt for token estimation+ # Store prompt for token estimation and raw messages for common_chat_logicself.spans[run_id].prompt_content = prompt_strself.spans[run_id].input_tokens = general_tokens(prompt_str)+ self.spans[run_id].input_messages_raw = messages# Set prompt attribute if capturing contentif self._capture_message_content:@@ -1050,7 +931,7 @@ def _create_callback_handler_class(# Store structured messages for event emissiontry:- structured_msgs = build_input_messages_from_langchain(messages)+ structured_msgs = build_input_messages(messages)self.spans[run_id].input_messages = structured_msgsexcept Exception as msg_err:logger.debug("Error building structured messages: %s", msg_err)@@ -1155,35 +1036,26 @@ def _create_callback_handler_class(if is_streaming:ttft = calculate_ttft(holder.token_timestamps, holder.start_time)tbt = calculate_tbt(holder.token_timestamps)- span.set_attribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft)- span.set_attribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt)- span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_IS_STREAM, True- )- else:- span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_IS_STREAM, False- )- # Extract response content+ # Extract response content and token usageinput_tokens = holder.input_tokensoutput_tokens = 0- completion_content = ""- model_name = holder.model_name-- # Get completion from streaming content firstcompletion_content = self._join_streaming_content(holder.streaming_content)+ model_name = holder.model_name- # Extract from response.generations (works for streaming and non-streaming)input_tokens, output_tokens, completion_content = (self._extract_from_generations(- response, input_tokens, output_tokens, completion_content+ holder,+ response,+ input_tokens,+ output_tokens,+ completion_content,))- # Try to get token usage from llm_output as another fallback+ # Handle token usage including reasoning tokens and cached tokensif response.llm_output:token_usage = response.llm_output.get("token_usage"@@ -1199,16 +1071,30 @@ def _create_callback_handler_class(or token_usage.get("output_tokens")or output_tokens)+ # OpenAI-style: prompt_tokens_details / input_tokens_details+ prompt_details = (+ token_usage.get("prompt_tokens_details")+ or token_usage.get("input_tokens_details")+ or {}+ )+ cached = prompt_details.get("cached_tokens", 0) or 0+ input_details = token_usage.get("input_tokens_details") or {}+ creation = input_details.get("cache_creation_tokens", 0) or 0+ # LangChain streaming: usage often in llm_output with input_token_details+ langchain_input = token_usage.get("input_token_details") or {}+ if cached == 0:+ cached = langchain_input.get("cache_read", 0) or 0+ if creation == 0:+ creation = langchain_input.get("cache_creation", 0) or 0+ holder.cache_read_input_tokens = cached+ holder.cache_creation_input_tokens = creation- # Calculate tokens from content if not available from metadata- # (Similar to OpenAI instrumentation approach)try:if output_tokens == 0 and completion_content:output_tokens = general_tokens(completion_content)except Exception as token_err:logger.debug("Error calculating output tokens: %s", token_err)- # If we still don't have input tokens, estimate from prompttry:if (input_tokens == 0@@ -1219,117 +1105,53 @@ def _create_callback_handler_class(except Exception as input_err:logger.debug("Error calculating input tokens: %s", input_err)- # Set completion content attribute- if completion_content and self._capture_message_content:- span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES,- completion_content[:5000],- )-- # Emit OTel log event- if self._event_provider and self._capture_message_content:- try:- # Get stored input messages- input_msgs = (- holder.input_messages if holder.input_messages else []- )-- # Build output messages- output_msgs = build_output_messages_langchain(- completion_content- )-- # Extract model parameters- model_params = (- holder.model_parameters if holder.model_parameters else {}- )-- # Determine response model- response_model = model_name- if response.llm_output:- response_model = response.llm_output.get(- "model_name"- ) or response.llm_output.get("model", model_name)-- emit_inference_event_langchain(- event_provider=self._event_provider,- operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- request_model=model_name,- response_model=response_model,- input_messages=input_msgs,- output_messages=output_msgs,- server_address="localhost",- server_port=8080,- temperature=model_params.get("temperature"),- max_tokens=model_params.get("max_tokens"),- top_p=model_params.get("top_p"),- top_k=model_params.get("top_k"),- frequency_penalty=model_params.get("frequency_penalty"),- presence_penalty=model_params.get("presence_penalty"),- input_tokens=input_tokens,- output_tokens=output_tokens,- )- except Exception as event_err:- logger.warning(- "Failed to emit inference event: %s",- event_err,- exc_info=True,- )-- # Always set token attributes- span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens- )- span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens+ response_model = model_name+ if response.llm_output:+ response_model = response.llm_output.get(+ "model_name"+ ) or response.llm_output.get("model", model_name)++ # Build scope and run common_chat_logic (OTel span/event/metrics)+ scope = type("Scope", (), {})()+ scope._span = span+ scope._kwargs = holder.model_parameters or {}+ scope._model_parameters = holder.model_parameters or {}+ scope._start_time = holder.start_time+ scope._end_time = end_time+ scope._server_address = "localhost"+ scope._server_port = 8080+ scope._response_model = response_model+ scope._response_id = (response.llm_output or {}).get("id") or None+ scope._llmresponse = completion_content or ""+ scope._finish_reason = "stop"+ scope._tools = None+ scope._input_tokens = input_tokens+ scope._output_tokens = output_tokens+ scope._cache_read_input_tokens = getattr(+ holder, "cache_read_input_tokens", 0)- span.set_attribute(- SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS,- input_tokens + output_tokens,+ scope._cache_creation_input_tokens = getattr(+ holder, "cache_creation_input_tokens", 0)-- # Calculate cost- cost = get_chat_model_cost(- model_name, self._pricing_info, input_tokens, output_tokens+ scope._timestamps = holder.token_timestamps+ scope._tbt = tbt+ scope._ttft = ttft+ scope._request_model = model_name+ scope._input_messages_raw = getattr(holder, "input_messages_raw", None)+ scope._prompts = getattr(holder, "prompts", [])++ common_chat_logic(+ scope,+ self._pricing_info,+ self._environment,+ self._application_name,+ self._metrics,+ self._capture_message_content,+ self._disable_metrics,+ self._version,+ is_streaming,+ self._event_provider,)- span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Set response model (if not already set in event emission above)- if "response_model" not in locals():- response_model = model_name- if response.llm_output:- response_model = response.llm_output.get(- "model_name"- ) or response.llm_output.get("model", model_name)-- if response.llm_output:- span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_MODEL, response_model- )-- # Record metrics (like OpenAI instrumentation)- if not self._disable_metrics and self._metrics:- try:- record_completion_metrics(- metrics=self._metrics,- gen_ai_operation=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- GEN_AI_PROVIDER_NAME=SemanticConvention.GEN_AI_SYSTEM_LANGCHAIN,- server_address="localhost",- server_port=8080,- request_model=model_name,- response_model=response_model,- environment=self._environment,- application_name=self._application_name,- start_time=holder.start_time,- end_time=end_time,- input_tokens=input_tokens,- output_tokens=output_tokens,- cost=cost,- tbt=tbt,- ttft=ttft,- )- except Exception as e:- logger.debug("Error recording LLM metrics: %s", e)self._end_span(run_id)diff --git a/sdk/python/src/openlit/instrumentation/langchain/utils.py b/sdk/python/src/openlit/instrumentation/langchain/utils.pynew file mode 100644index 00000000..dde958e4--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/langchain/utils.py@@ -0,0 +1,560 @@+"""+LangChain OpenTelemetry instrumentation utility functions.+"""++import json+import logging+import time+from typing import Any, List++from opentelemetry.trace import Status, StatusCode++from openlit.__helpers import (+ common_span_attributes,+ get_chat_model_cost,+ record_completion_metrics,+ calculate_tbt,+ general_tokens,+ otel_event,+)+from openlit.semcov import SemanticConvention++logger = logging.getLogger(__name__)+++def format_content(messages_or_prompts: Any) -> str:+ """+ Format messages or prompts for display and token fallback.+ Handles LangChain message lists (list-of-lists of BaseMessage) and prompt lists.+ """+ if not messages_or_prompts:+ return ""+ if isinstance(messages_or_prompts, str):+ return messages_or_prompts+ parts = []+ for item in messages_or_prompts:+ if isinstance(item, str):+ parts.append(item)+ continue+ if isinstance(item, list):+ for msg in item:+ role = getattr(msg, "type", getattr(msg, "role", "user"))+ content = getattr(msg, "content", str(msg))+ parts.append(f"{role}: {content}")+ else:+ parts.append(str(item))+ return "\n".join(parts) if parts else ""+++def build_input_messages_from_langchain(messages: List) -> List[dict]:+ """+ Convert LangChain messages (list-of-lists of BaseMessage) to OTel input message structure.+ """+ try:+ structured = []+ for msg_list in messages:+ for msg in msg_list:+ role = getattr(msg, "type", "user")+ content = getattr(msg, "content", str(msg))+ role_mapping = {+ "system": "system",+ "human": "user",+ "ai": "assistant",+ "tool": "tool",+ "function": "tool",+ }+ otel_role = role_mapping.get(role, "user")+ structured.append(+ {+ "role": otel_role,+ "parts": [{"type": "text", "content": str(content)}],+ }+ )+ return structured+ except Exception as e:+ logger.debug("Error building input messages from LangChain: %s", e)+ return []+++def build_input_messages_from_prompts(prompts: List) -> List[dict]:+ """+ Convert LangChain prompts (strings) to OTel input message structure.+ """+ try:+ structured = []+ for prompt_list in prompts:+ prompt_text = (+ prompt_list if isinstance(prompt_list, str) else str(prompt_list)+ )+ structured.append(+ {+ "role": "user",+ "parts": [{"type": "text", "content": prompt_text}],+ }+ )+ return structured+ except Exception as e:+ logger.debug("Error building input messages from prompts: %s", e)+ return []+++def build_input_messages(messages_or_prompts: Any) -> List[dict]:+ """+ Convert LangChain request messages or prompts to OTel input message structure.+ Follows gen-ai-input-messages schema.+ Dispatches to from_langchain (list-of-lists of messages) or from_prompts (list of strings).+ """+ if not messages_or_prompts:+ return []+ # Prompts: list of strings or list of lists with string at 0+ if isinstance(messages_or_prompts, list) and messages_or_prompts:+ first = messages_or_prompts[0]+ if isinstance(first, str):+ return build_input_messages_from_prompts(messages_or_prompts)+ if isinstance(first, list) and first and hasattr(first[0], "content"):+ return build_input_messages_from_langchain(messages_or_prompts)+ return build_input_messages_from_prompts(messages_or_prompts)+ return []+++def build_output_messages(+ response_text: str,+ finish_reason: str = "stop",+ tool_calls: Any = None,+) -> List[dict]:+ """+ Convert LangChain response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ try:+ if not response_text and not tool_calls:+ return []+ parts = []+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls:+ for tc in tool_calls if isinstance(tool_calls, list) else [tool_calls]:+ if isinstance(tc, dict):+ func = tc.get("function", tc.get("function_call", {}))+ if isinstance(func, dict):+ parts.append(+ {+ "type": "tool_call",+ "id": tc.get("id", ""),+ "name": func.get("name", ""),+ "arguments": func.get(+ "arguments", func.get("args", {})+ ),+ }+ )+ if not parts:+ return []+ return [+ {+ "role": "assistant",+ "parts": parts,+ "finish_reason": finish_reason,+ }+ ]+ except Exception as e:+ logger.debug("Error building output messages: %s", e)+ return []+++def _set_span_messages_as_array(+ span: Any,+ input_messages: Any,+ output_messages: Any,+) -> None:+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider: Any,+ operation_name: str,+ request_model: str,+ response_model: str,+ input_messages: Any = None,+ output_messages: Any = None,+ tool_definitions: Any = None,+ server_address: str = None,+ server_port: Any = None,+ **extra_attrs: Any,+) -> None:+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return++ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port is not None:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions++ for key, value in extra_attrs.items():+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "top_k":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_K] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "reasoning_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_REASONING_TOKENS] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value++ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)++ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)+++def _get_scope_params(scope: Any) -> dict:+ """Get request params from scope (_kwargs or _model_parameters)."""+ return (+ getattr(scope, "_kwargs", None)+ or getattr(scope, "_model_parameters", None)+ or {}+ )+++def common_chat_logic(+ scope: Any,+ pricing_info: Any,+ environment: str,+ application_name: str,+ metrics: Any,+ capture_message_content: bool,+ disable_metrics: bool,+ version: str,+ is_stream: bool,+ event_provider: Any = None,+) -> None:+ """+ Process chat request and generate Telemetry.+ """+ scope._end_time = time.time()+ if len(scope._timestamps) > 1:+ scope._tbt = calculate_tbt(scope._timestamps)++ prompt = format_content(+ getattr(scope, "_input_messages_raw", None)+ or getattr(scope, "_prompts", None)+ or []+ )+ request_model = getattr(scope, "_request_model", None) or "unknown"++ if (+ hasattr(scope, "_input_tokens")+ and scope._input_tokens is not None+ and hasattr(scope, "_output_tokens")+ and scope._output_tokens is not None+ ):+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)++ # Common Span Attributes+ common_span_attributes(+ scope,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_LANGCHAIN,+ scope._server_address,+ scope._server_port,+ request_model,+ scope._response_model,+ environment,+ application_name,+ is_stream,+ scope._tbt,+ scope._ttft,+ version,+ )++ # Span Attributes for Request parameters+ params = _get_scope_params(scope)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,+ params.get("temperature", 1.0),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,+ params.get("max_tokens", params.get("max_completion_tokens", -1)),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_TOP_P,+ params.get("top_p", 1.0),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,+ params.get("frequency_penalty", 0.0),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,+ params.get("presence_penalty", 0.0),+ )+ stop = params.get("stop", params.get("stop_sequences", []))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,+ stop if isinstance(stop, list) else [stop] if stop else [],+ )+ seed_val = params.get("seed")+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_SEED,+ int(seed_val) if seed_val is not None else 0,+ )++ # Span Attributes for Response parameters+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [getattr(scope, "_finish_reason", "stop")],+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_TYPE,+ "text" if isinstance(scope._llmresponse, str) else "json",+ )++ # Span Attributes for Tools+ if getattr(scope, "_tools", None):+ tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]+ names, ids, args = (+ zip(+ *[+ (+ t.get("function", {}).get("name", t.get("name", "")),+ str(t.get("id", "")),+ str(+ t.get("function", {}).get(+ "arguments", t.get("arguments", "")+ )+ ),+ )+ for t in tools+ if isinstance(t, dict) and t+ ]+ )+ if tools+ else ([], [], [])+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, ", ".join(filter(None, names))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, ", ".join(filter(None, ids))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS, ", ".join(filter(None, args))+ )++ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # Reasoning tokens (optional)+ if (+ hasattr(scope, "_reasoning_tokens")+ and scope._reasoning_tokens+ and scope._reasoning_tokens > 0+ ):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_REASONING_TOKENS, scope._reasoning_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens + scope._reasoning_tokens,+ )++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )++ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_raw = (+ getattr(scope, "_input_messages_raw", None)+ or getattr(scope, "_prompts", None)+ or []+ )+ input_msgs = build_input_messages(input_raw)+ output_msgs = build_output_messages(+ scope._llmresponse,+ getattr(scope, "_finish_reason", "stop"),+ tool_calls=getattr(scope, "_tools", None),+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [getattr(scope, "_finish_reason", "stop")],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": params.get("temperature"),+ "max_tokens": params.get(+ "max_tokens", params.get("max_completion_tokens")+ ),+ "top_p": params.get("top_p"),+ "frequency_penalty": params.get("frequency_penalty"),+ "presence_penalty": params.get("presence_penalty"),+ "stop_sequences": params.get("stop", params.get("stop_sequences")),+ "seed": params.get("seed"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ if (+ hasattr(scope, "_system_instructions")+ and scope._system_instructions+ ):+ extra["system_instructions"] = scope._system_instructions+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=scope._response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++ scope._span.set_status(Status(StatusCode.OK))++ # Record metrics+ if not disable_metrics and metrics:+ try:+ record_completion_metrics(+ metrics,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_LANGCHAIN,+ scope._server_address,+ scope._server_port,+ request_model,+ scope._response_model,+ environment,+ application_name,+ scope._start_time,+ scope._end_time,+ input_tokens,+ output_tokens,+ cost,+ scope._tbt,+ scope._ttft,+ )+ except Exception as e:+ logger.debug("Error recording completion metrics: %s", e)diff --git a/sdk/python/src/openlit/instrumentation/litellm/async_litellm.py b/sdk/python/src/openlit/instrumentation/litellm/async_litellm.pyindex b266cc71..0ed38e2b 100644--- a/sdk/python/src/openlit/instrumentation/litellm/async_litellm.py+++ b/sdk/python/src/openlit/instrumentation/litellm/async_litellm.py@@ -56,6 +56,8 @@ def acompletion(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsself._start_time = time.time()diff --git a/sdk/python/src/openlit/instrumentation/litellm/litellm.py b/sdk/python/src/openlit/instrumentation/litellm/litellm.pyindex 888b5d6d..05019cec 100644--- a/sdk/python/src/openlit/instrumentation/litellm/litellm.py+++ b/sdk/python/src/openlit/instrumentation/litellm/litellm.py@@ -56,6 +56,8 @@ def completion(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargsself._start_time = time.time()diff --git a/sdk/python/src/openlit/instrumentation/litellm/utils.py b/sdk/python/src/openlit/instrumentation/litellm/utils.pyindex 95223801..74835662 100644--- a/sdk/python/src/openlit/instrumentation/litellm/utils.py+++ b/sdk/python/src/openlit/instrumentation/litellm/utils.py@@ -2,6 +2,8 @@LiteLLM OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -19,6 +21,8 @@ from openlit.__helpers import ()from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -46,13 +50,8 @@ def format_content(messages):def build_input_messages(messages):"""- Convert LiteLLM messages (OpenAI-compatible format) to OTel input message structure.-- Args:- messages: List of message objects from LiteLLM request-- Returns:- List of ChatMessage objects following OTel schema+ Convert LiteLLM request messages to OTel input message structure.+ Follows gen-ai-input-messages schema."""structured_messages = []@@ -111,14 +110,7 @@ def build_input_messages(messages):def build_output_messages(response_text, finish_reason, tool_calls=None):"""Convert LiteLLM response to OTel output message structure.-- Args:- response_text: The text response- finish_reason: LiteLLM finish reason value- tool_calls: Tool calls from response-- Returns:- List with single OutputMessage following OTel schema+ Follows gen-ai-output-messages schema."""parts = []@@ -166,6 +158,27 @@ def build_tool_definitions(tools):return tools if tools else None+def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -179,87 +192,80 @@ def emit_inference_event(**extra_attrs,):"""- Centralized function to emit gen_ai.client.inference.operation.details event.-- Args:- event_provider: The OTel event provider- operation_name: Operation type (chat, completion, embedding, etc.)- request_model: Model specified in request- response_model: Model returned in response- input_messages: Structured input messages- output_messages: Structured output messages- tool_definitions: Tool/function definitions- server_address: API server address- server_port: API server port- **extra_attrs: Additional attributes (temperature, max_tokens, etc.)+ Emit gen_ai.client.inference.operation.details event."""try:if not event_provider:return- # Build base attributes- attributes = {- SemanticConvention.GEN_AI_OPERATION_NAME: operation_name,- }+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}- # Add model attributesif request_model:attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_modelif response_model:attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model- # Add server attributesif server_address:- attributes["server.address"] = server_address+ attributes[SemanticConvention.SERVER_ADDRESS] = server_addressif server_port:- attributes["server.port"] = server_port+ attributes[SemanticConvention.SERVER_PORT] = server_port- # Add messages (only if not None/empty)- if input_messages:+ if input_messages is not None:attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages- if output_messages:+ if output_messages is not None:attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages-- # Add tool definitions (recommended)- if tool_definitions:+ if tool_definitions is not None:attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions- # Map extra attributes to semantic conventions- attr_mapping = {- "temperature": SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- "max_tokens": SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,- "top_p": SemanticConvention.GEN_AI_REQUEST_TOP_P,- "frequency_penalty": SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- "presence_penalty": SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,- "stop_sequences": SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,- "seed": SemanticConvention.GEN_AI_REQUEST_SEED,- "response_id": SemanticConvention.GEN_AI_RESPONSE_ID,- "finish_reasons": SemanticConvention.GEN_AI_RESPONSE_FINISH_REASONS,- "choice_count": SemanticConvention.GEN_AI_RESPONSE_CHOICE_COUNT,- "output_type": SemanticConvention.GEN_AI_RESPONSE_OUTPUT_TYPE,- "input_tokens": SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,- "output_tokens": SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,- "cache_creation_input_tokens": SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,- "cache_read_input_tokens": SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,- }-for key, value in extra_attrs.items():- if value is not None and key in attr_mapping:- attributes[attr_mapping[key]] = value+ if key == "response_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type" and value is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n") and value is not None and value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value if value is not None else 0+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value if value is not None else 0+ elif key == "error_type" and value is not None:+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions" and value is not None:+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "conversation_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value- # Create and emit eventevent = otel_event(name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,attributes=attributes,- body="", # Per spec, all data in attributes+ body="",)-event_provider.emit(event)-except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)@@ -316,8 +322,17 @@ def process_chunk(scope, chunk):)if chunked.get("usage"):- scope._input_tokens = chunked.get("usage").get("prompt_tokens", 0)- scope._output_tokens = chunked.get("usage").get("completion_tokens", 0)+ usage = chunked.get("usage", {})+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ prompt_tokens_details = usage.get(+ "prompt_tokens_details", usage.get("input_tokens_details", {})+ )+ scope._cache_read_input_tokens = prompt_tokens_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = usage.get(+ "completion_tokens_details", {}+ ).get("cached_tokens", 0)scope._response_id = chunked.get("id")scope._response_model = chunked.get("model")finish_reason = chunked.get("choices", [{}])[0].get("finish_reason")@@ -344,10 +359,10 @@ def common_chat_logic(Process chat request and generate Telemetry"""+ scope._end_time = getattr(scope, "_end_time", None) or time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "openai/gpt-4o")cost = get_chat_model_cost(@@ -371,7 +386,6 @@ def common_chat_logic(version,)- # Helper function to handle None values with proper defaultsdef safe_get(value, default):return default if value is None else value@@ -427,20 +441,7 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)- # Span Attributes for Cost and Tokens- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,- )- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Span Attributes for Tools - optimized+ # Span Attributes for Toolsif scope._tools:tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]@@ -470,22 +471,65 @@ def common_chat_logic(SemanticConvention.GEN_AI_TOOL_ARGS, ", ".join(filter(None, args)))- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ scope._input_tokens + scope._output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # Emit OTel log event+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, scope._tools+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ # Emit inference eventif event_provider:try:- input_msgs = build_input_messages(scope._kwargs.get("messages", []))- output_msgs = build_output_messages(- scope._llmresponse, scope._finish_reason, scope._tools- )tool_defs = build_tool_definitions(scope._kwargs.get("tools"))-+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop"),+ "seed": scope._kwargs.get("seed"),+ "input_tokens": scope._input_tokens,+ "output_tokens": scope._output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,@@ -494,27 +538,16 @@ def common_chat_logic(input_messages=input_msgs,output_messages=output_msgs,tool_definitions=tool_defs,- server_address="api.litellm.ai",- server_port=443,- response_id=scope._response_id,- finish_reasons=[scope._finish_reason],- temperature=scope._kwargs.get("temperature"),- max_tokens=scope._kwargs.get("max_tokens"),- top_p=scope._kwargs.get("top_p"),- frequency_penalty=scope._kwargs.get("frequency_penalty"),- presence_penalty=scope._kwargs.get("presence_penalty"),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_completion_metrics(metrics,@@ -597,8 +630,17 @@ def process_chat_response((choice.get("message", {}).get("content") or "")for choice in response_dict.get("choices", []))- scope._input_tokens = response_dict.get("usage", {}).get("prompt_tokens", 0)- scope._output_tokens = response_dict.get("usage", {}).get("completion_tokens", 0)+ usage = response_dict.get("usage", {})+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ prompt_tokens_details = usage.get(+ "prompt_tokens_details", usage.get("input_tokens_details", {})+ )+ scope._cache_read_input_tokens = prompt_tokens_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = usage.get("completion_tokens_details", {}).get(+ "cached_tokens", 0+ )scope._response_id = response_dict.get("id")scope._response_model = response_dict.get("model")scope._finish_reason = str(@@ -736,17 +778,14 @@ def process_embedding_response(request_model=request_model,response_model=scope._response_model,input_messages=input_msgs,- output_messages=[], # Embeddings don't have text output+ output_messages=[],tool_definitions=None,- server_address="api.litellm.ai",- server_port=443,+ server_address=scope._server_address,+ server_port=scope._server_port,response_id=None,input_tokens=scope._input_tokens,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))diff --git a/sdk/python/src/openlit/instrumentation/mistral/__init__.py b/sdk/python/src/openlit/instrumentation/mistral/__init__.pyindex 8c23ff33..d8e5131d 100644--- a/sdk/python/src/openlit/instrumentation/mistral/__init__.py+++ b/sdk/python/src/openlit/instrumentation/mistral/__init__.py@@ -31,6 +31,7 @@ class MistralInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("mistralai")# sync chat completions@@ -46,6 +47,7 @@ class MistralInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -62,6 +64,7 @@ class MistralInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -94,6 +97,7 @@ class MistralInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -110,6 +114,7 @@ class MistralInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/mistral/async_mistral.py b/sdk/python/src/openlit/instrumentation/mistral/async_mistral.pyindex 586ac422..7ce46ae5 100644--- a/sdk/python/src/openlit/instrumentation/mistral/async_mistral.py+++ b/sdk/python/src/openlit/instrumentation/mistral/async_mistral.py@@ -26,6 +26,7 @@ def async_complete(capture_message_content,metrics,disable_metrics,+ event_provider,):"""Generates a telemetry wrapper for GenAI complete function call@@ -60,6 +61,7 @@ def async_complete(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)@@ -77,6 +79,7 @@ def async_stream(capture_message_content,metrics,disable_metrics,+ event_provider,):"""Generates a telemetry wrapper for GenAI stream function call@@ -107,6 +110,8 @@ def async_stream(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargs@@ -151,6 +156,7 @@ def async_stream(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:diff --git a/sdk/python/src/openlit/instrumentation/mistral/mistral.py b/sdk/python/src/openlit/instrumentation/mistral/mistral.pyindex da8320dc..3808f0b8 100644--- a/sdk/python/src/openlit/instrumentation/mistral/mistral.py+++ b/sdk/python/src/openlit/instrumentation/mistral/mistral.py@@ -26,6 +26,7 @@ def complete(capture_message_content,metrics,disable_metrics,+ event_provider,):"""Generates a telemetry wrapper for GenAI complete function call@@ -60,6 +61,7 @@ def complete(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)@@ -77,6 +79,7 @@ def stream(capture_message_content,metrics,disable_metrics,+ event_provider,):"""Generates a telemetry wrapper for GenAI stream function call@@ -107,6 +110,8 @@ def stream(self._tools = Noneself._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._args = argsself._kwargs = kwargs@@ -151,6 +156,7 @@ def stream(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:diff --git a/sdk/python/src/openlit/instrumentation/mistral/utils.py b/sdk/python/src/openlit/instrumentation/mistral/utils.pyindex 680bb74e..80fa2cd0 100644--- a/sdk/python/src/openlit/instrumentation/mistral/utils.py+++ b/sdk/python/src/openlit/instrumentation/mistral/utils.py@@ -2,6 +2,8 @@Mistral OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -13,11 +15,14 @@ from openlit.__helpers import (get_chat_model_cost,get_embed_model_cost,common_span_attributes,+ otel_event,record_completion_metrics,record_embedding_metrics,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -43,6 +48,228 @@ def format_content(messages):return "\n".join(formatted_messages)+def build_input_messages(messages):+ """+ Convert Mistral request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not messages:+ return []+ if isinstance(messages, str):+ return [{"role": "user", "parts": [{"type": "text", "content": messages}]}]+ otel_messages = []+ for message in messages:+ try:+ role = message.get("role", "user")+ content = message.get("content", "")+ parts = []+ if isinstance(content, list):+ for item in content:+ if not isinstance(item, dict):+ continue+ item_type = item.get("type")+ if item_type == "text":+ text_content = item.get("text", "")+ if text_content:+ parts.append({"type": "text", "content": text_content})+ elif item_type == "image_url":+ image_url_obj = item.get("image_url", {})+ url = (+ image_url_obj.get("url", "")+ if isinstance(image_url_obj, dict)+ else ""+ )+ if url and not url.startswith("data:"):+ parts.append(+ {"type": "uri", "modality": "image", "uri": url}+ )+ elif isinstance(content, str) and content:+ parts.append({"type": "text", "content": content})+ if parts:+ otel_messages.append({"role": role, "parts": parts})+ except Exception as e:+ logger.warning("Failed to process input message: %s", e, exc_info=True)+ continue+ return otel_messages+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Mistral response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ try:+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls and isinstance(tool_calls, list):+ for tool_call in tool_calls:+ try:+ if isinstance(tool_call, dict):+ func = tool_call.get("function", {})+ tool_id = tool_call.get("id", "")+ tool_name = func.get("name", "")+ tool_args = func.get("arguments", {})+ else:+ tool_id = getattr(tool_call, "id", "")+ func = getattr(tool_call, "function", {}) or {}+ tool_name = (+ func.get("name", "")+ if isinstance(func, dict)+ else getattr(func, "name", "")+ )+ tool_args = (+ func.get("arguments", "")+ if isinstance(func, dict)+ else getattr(func, "arguments", "")+ )+ if isinstance(tool_args, str):+ try:+ tool_args = json.loads(tool_args)+ except Exception:+ tool_args = {"raw": tool_args}+ parts.append(+ {+ "type": "tool_call",+ "id": tool_id,+ "name": tool_name,+ "arguments": tool_args,+ }+ )+ except Exception as e:+ logger.warning("Failed to process tool call: %s", e, exc_info=True)+ finish_reason_map = {+ "stop": "stop",+ "length": "length",+ "content_filter": "content_filter",+ "tool_calls": "tool_call",+ }+ otel_finish_reason = finish_reason_map.get(+ finish_reason, finish_reason or "stop"+ )+ return [+ {"role": "assistant", "parts": parts, "finish_reason": otel_finish_reason}+ ]+ except Exception as e:+ logger.warning("Failed to build output messages: %s", e, exc_info=True)+ return [{"role": "assistant", "parts": [], "finish_reason": "stop"}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def process_chunk(scope, chunk):"""Process a chunk of response data and update state.@@ -99,14 +326,25 @@ def process_chunk(scope, chunk):# Handle usage information (typically only in final chunk)if data.get("usage"):- scope._input_tokens = data.get("usage").get("prompt_tokens", 0)- scope._output_tokens = data.get("usage").get("completion_tokens", 0)+ usage = data.get("usage")+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)scope._response_id = data.get("id")scope._response_model = data.get("model")scope._finish_reason = (choices[0].get("finish_reason", "") if choices else "")scope._end_time = time.time()+ # Handle token usage including reasoning tokens and cached tokens+ input_tokens_details = usage.get("input_tokens_details") or usage.get(+ "prompt_tokens_details", {}+ )+ scope._cache_read_input_tokens = input_tokens_details.get(+ "cached_tokens", 0+ )+ scope._cache_creation_input_tokens = usage.get(+ "cache_creation_input_tokens", 0+ )def common_chat_logic(@@ -119,17 +357,19 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""- Process chat request and generate Telemetry+ Process chat request and generate Telemetry."""+ scope._end_time = time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "mistral-small-latest")+ # Compute costcost = get_chat_model_cost(request_model, pricing_info, scope._input_tokens, scope._output_tokens)@@ -151,6 +391,8 @@ def common_chat_logic(version,)+ # Mistral chat (no additional API type attribute)+# Span Attributes for Request parametersscope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED, scope._kwargs.get("seed", "")@@ -192,20 +434,7 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)- # Span Attributes for Cost and Tokens- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,- )- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Span Attributes for Tools - optimized+ # Span Attributes for Toolsif scope._tools:tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]@@ -235,30 +464,80 @@ def common_chat_logic(SemanticConvention.GEN_AI_TOOL_ARGS, ", ".join(filter(None, args)))- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse- )+ # Span Attributes for Cost and Tokens+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ scope._input_tokens + scope._output_tokens,+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)+ # Span Attributes for Content+ if capture_message_content:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, scope._tools+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature", 0.3),+ "max_tokens": scope._kwargs.get("max_tokens", -1),+ "top_p": scope._kwargs.get("p", 1.0),+ "frequency_penalty": scope._kwargs.get("frequency_penalty", 0.0),+ "presence_penalty": scope._kwargs.get("presence_penalty", 0.0),+ "stop_sequences": scope._kwargs.get("stop_sequences", []),+ "seed": scope._kwargs.get("seed", ""),+ "input_tokens": scope._input_tokens,+ "output_tokens": scope._output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=scope._response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++ # Span status and metricsscope._span.set_status(Status(StatusCode.OK))-- # Metricsif not disable_metrics:record_completion_metrics(metrics,@@ -289,6 +568,7 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process streaming chat request and generate Telemetry@@ -304,6 +584,7 @@ def process_streaming_chat_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -321,6 +602,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -360,6 +642,14 @@ def process_chat_response(else:scope._tools = None+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage", {})+ input_tokens_details = usage.get("input_tokens_details") or usage.get(+ "prompt_tokens_details", {}+ )+ scope._cache_read_input_tokens = input_tokens_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = usage.get("cache_creation_input_tokens", 0)+common_chat_logic(scope,pricing_info,@@ -370,6 +660,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/ollama/async_ollama.py b/sdk/python/src/openlit/instrumentation/ollama/async_ollama.pyindex cf3837e4..331199d1 100644--- a/sdk/python/src/openlit/instrumentation/ollama/async_ollama.py+++ b/sdk/python/src/openlit/instrumentation/ollama/async_ollama.py@@ -55,6 +55,8 @@ def async_chat(self._tools = []self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._response_role = ""self._span_name = span_nameself._args = args@@ -146,7 +148,6 @@ def async_chat(try:response = process_chat_response(response=response,- gen_ai_endpoint="ollama.chat",pricing_info=pricing_info,server_port=server_port,server_address=server_address,@@ -209,6 +210,8 @@ def async_generate(self._tools = []self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._response_role = ""self._span_name = span_nameself._args = args@@ -300,7 +303,6 @@ def async_generate(try:response = process_generate_response(response=response,- gen_ai_endpoint="ollama.generate",pricing_info=pricing_info,server_port=server_port,server_address=server_address,diff --git a/sdk/python/src/openlit/instrumentation/ollama/ollama.py b/sdk/python/src/openlit/instrumentation/ollama/ollama.pyindex 8ae613c7..21efc97f 100644--- a/sdk/python/src/openlit/instrumentation/ollama/ollama.py+++ b/sdk/python/src/openlit/instrumentation/ollama/ollama.py@@ -55,6 +55,8 @@ def chat(self._tools = []self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._response_role = ""self._span_name = span_nameself._args = args@@ -146,7 +148,6 @@ def chat(try:response = process_chat_response(response=response,- gen_ai_endpoint="ollama.chat",pricing_info=pricing_info,server_port=server_port,server_address=server_address,@@ -209,6 +210,8 @@ def generate(self._tools = []self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._response_role = ""self._span_name = span_nameself._args = args@@ -300,7 +303,6 @@ def generate(try:response = process_generate_response(response=response,- gen_ai_endpoint="ollama.generate",pricing_info=pricing_info,server_port=server_port,server_address=server_address,diff --git a/sdk/python/src/openlit/instrumentation/ollama/utils.py b/sdk/python/src/openlit/instrumentation/ollama/utils.pyindex 46299aa4..3b1fa87a 100644--- a/sdk/python/src/openlit/instrumentation/ollama/utils.py+++ b/sdk/python/src/openlit/instrumentation/ollama/utils.py@@ -2,6 +2,8 @@Ollama OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -20,6 +22,8 @@ from openlit.__helpers import ()from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -47,8 +51,8 @@ def format_content(messages):def build_input_messages(messages):"""- Convert Ollama messages to OTel input message structure.- Ollama uses OpenAI-compatible message format.+ Convert Ollama request messages to OTel input message structure.+ Follows gen-ai-input-messages schema."""structured_messages = []for msg in messages:@@ -100,7 +104,7 @@ def build_input_messages(messages):def build_output_messages(response_text, finish_reason, tool_calls=None):"""Convert Ollama response to OTel output message structure.- Maps Ollama finish reasons to OTel standard.+ Follows gen-ai-output-messages schema."""parts = []if response_text:@@ -137,6 +141,27 @@ def build_tool_definitions(tools):return tools if tools else None+def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -150,46 +175,73 @@ def emit_inference_event(**extra_attrs,):"""- Centralized function to emit gen_ai.client.inference.operation.details event.- Uses Ollama-specific defaults (localhost:11434).+ Emit gen_ai.client.inference.operation.details event."""try:if not event_provider:return- attributes = {SemanticConvention.GEN_AI_OPERATION_NAME: operation_name}+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}if request_model:attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_modelif response_model:attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_modelif server_address:- attributes["server.address"] = server_address+ attributes[SemanticConvention.SERVER_ADDRESS] = server_addressif server_port:- attributes["server.port"] = server_port- if input_messages:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages- if output_messages:+ if output_messages is not None:attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages- if tool_definitions:+ if tool_definitions is not None:attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions- # Map extra attributes to semantic conventions- attr_mapping = {- "response_id": SemanticConvention.GEN_AI_RESPONSE_ID,- "finish_reasons": SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,- "temperature": SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- "max_tokens": SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,- "top_p": SemanticConvention.GEN_AI_REQUEST_TOP_P,- "top_k": SemanticConvention.GEN_AI_REQUEST_TOP_K,- "repeat_penalty": SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- "input_tokens": SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,- "output_tokens": SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,- }-+ # Map extra attributes to semantic conventions (include cache tokens even when 0)for key, value in extra_attrs.items():- if value is not None and key in attr_mapping:- attributes[attr_mapping[key]] = value+ if key == "response_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type" and value is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "top_k" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_K] = value+ elif key in ("frequency_penalty", "repeat_penalty") and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n") and value is not None and value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value if value is not None else 0+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value if value is not None else 0+ elif key == "error_type" and value is not None:+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions" and value is not None:+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "conversation_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = valueevent = otel_event(name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,@@ -198,9 +250,6 @@ def emit_inference_event()event_provider.emit(event)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)@@ -225,8 +274,11 @@ def process_chunk(self, chunk):if chunked.get("eval_count"):self._response_role = chunked.get("message", {}).get("role", "")+ # Handle token usage including reasoning tokens and cached tokensself._input_tokens = chunked.get("prompt_eval_count", 0)self._output_tokens = chunked.get("eval_count", 0)+ self._cache_read_input_tokens = 0 # Ollama does not expose cached tokens+ self._cache_creation_input_tokens = 0self._response_model = chunked.get("model", "")self._finish_reason = chunked.get("done_reason", "")@@ -269,7 +321,6 @@ def record_embedding_metrics(def common_chat_logic(scope,- gen_ai_endpoint,pricing_info,environment,application_name,@@ -277,6 +328,7 @@ def common_chat_logic(capture_message_content,disable_metrics,version,+ is_stream,event_provider=None,):"""@@ -287,10 +339,8 @@ def common_chat_logic(if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)json_body = scope._kwargs.get("json", {}) or {}- messages = json_body.get("messages", scope._kwargs.get("messages", ""))- prompt = format_content(messages)request_model = json_body.get("model") or scope._kwargs.get("model", "llama3.2")- is_stream = scope._kwargs.get("stream", False)+ options = json_body.get("options", scope._kwargs.get("options", {}))cost = get_chat_model_cost(request_model, pricing_info, scope._input_tokens, scope._output_tokens@@ -304,7 +354,7 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- request_model,+ getattr(scope, "_response_model", request_model),environment,application_name,is_stream,@@ -314,8 +364,7 @@ def common_chat_logic()# Span Attributes for Request parameters- options = json_body.get("options", scope._kwargs.get("options", {}))- attributes = [+ request_attrs = [(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, "repeat_penalty"),(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, "max_tokens"),(SemanticConvention.GEN_AI_REQUEST_SEED, "seed"),@@ -324,7 +373,7 @@ def common_chat_logic((SemanticConvention.GEN_AI_REQUEST_TOP_P, "top_p"),(SemanticConvention.GEN_AI_REQUEST_TOP_K, "top_k"),]- for attribute, key in attributes:+ for attribute, key in request_attrs:value = options.get(key)if value is not None:scope._span.set_attribute(attribute, value)@@ -338,6 +387,24 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if getattr(scope, "_tools", None):+ tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]+ tools = [t for t in tools if isinstance(t, dict) and t]+ if tools:+ names = [t.get("function", {}).get("name", "") for t in tools]+ ids = [str(t.get("id", "")) for t in tools]+ args = [str(t.get("function", {}).get("arguments", "")) for t in tools]+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, ", ".join(filter(None, names))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, ", ".join(filter(None, ids))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS, ", ".join(filter(None, args))+ )+# Span Attributes for Cost and Tokensscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens@@ -351,85 +418,72 @@ def common_chat_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Tools- if scope._tools is not None:- if isinstance(scope._tools, dict):- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("function", "")- ).get("name", "")- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS,- str(scope._tools.get("function", "").get("arguments", "")),- )- elif isinstance(scope._tools, list) and len(scope._tools) > 0:- for tool in scope._tools:- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME,- tool.get("function", {}).get("name", ""),- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(tool.get("id", ""))- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS,- str(tool.get("function", {}).get("arguments", "")),- )+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )- # Span Attributes for Content+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ input_msgs = build_input_messages(+ json_body.get("messages", scope._kwargs.get("messages", []))+ )+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, scope._tools)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)- # Emit OTel log event+ # Emit inference eventif event_provider:try:- json_body = scope._kwargs.get("json", {}) or {}- input_msgs = build_input_messages(- json_body.get("messages", scope._kwargs.get("messages", []))- )- output_msgs = build_output_messages(- scope._llmresponse, scope._finish_reason, scope._tools- )tool_defs = build_tool_definitions(json_body.get("tools", scope._kwargs.get("tools")))-- # Extract options for parameters- options = json_body.get("options", scope._kwargs.get("options", {}))-+ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": options.get("temperature"),+ "max_tokens": options.get("max_tokens"),+ "top_p": options.get("top_p"),+ "top_k": options.get("top_k"),+ "repeat_penalty": options.get("repeat_penalty"),+ "input_tokens": scope._input_tokens,+ "output_tokens": scope._output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,request_model=request_model,- response_model=scope._response_model,+ response_model=getattr(scope, "_response_model", request_model),input_messages=input_msgs,output_messages=output_msgs,tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- finish_reasons=[scope._finish_reason],- temperature=options.get("temperature"),- max_tokens=options.get("max_tokens"),- top_p=options.get("top_p"),- top_k=options.get("top_k"),- repeat_penalty=options.get("repeat_penalty"),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,+ **extra,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_completion_metrics(metrics,@@ -438,14 +492,14 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- scope._response_model,+ getattr(scope, "_response_model", request_model),environment,application_name,scope._start_time,scope._end_time,- cost,scope._input_tokens,scope._output_tokens,+ cost,scope._tbt,scope._ttft,)@@ -453,7 +507,6 @@ def common_chat_logic(def common_generate_logic(scope,- gen_ai_endpoint,pricing_info,environment,application_name,@@ -461,6 +514,7 @@ def common_generate_logic(capture_message_content,disable_metrics,version,+ is_stream,event_provider=None,):"""@@ -473,7 +527,7 @@ def common_generate_logic(json_body = scope._kwargs.get("json", {}) or {}prompt = json_body.get("prompt")request_model = json_body.get("model") or scope._kwargs.get("model", "llama3.2")- is_stream = scope._kwargs.get("stream", False)+ options = json_body.get("options", scope._kwargs.get("options", {}))cost = get_chat_model_cost(request_model, pricing_info, scope._input_tokens, scope._output_tokens@@ -487,7 +541,7 @@ def common_generate_logic(scope._server_address,scope._server_port,request_model,- request_model,+ getattr(scope, "_response_model", request_model),environment,application_name,is_stream,@@ -497,8 +551,7 @@ def common_generate_logic()# Span Attributes for Request parameters- options = json_body.get("options", scope._kwargs.get("options", {}))- attributes = [+ request_attrs = [(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, "repeat_penalty"),(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, "max_tokens"),(SemanticConvention.GEN_AI_REQUEST_SEED, "seed"),@@ -507,7 +560,7 @@ def common_generate_logic((SemanticConvention.GEN_AI_REQUEST_TOP_P, "top_p"),(SemanticConvention.GEN_AI_REQUEST_TOP_K, "top_k"),]- for attribute, key in attributes:+ for attribute, key in request_attrs:value = options.get(key)if value is not None:scope._span.set_attribute(attribute, value)@@ -521,6 +574,24 @@ def common_generate_logic("text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if getattr(scope, "_tools", None):+ tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]+ tools = [t for t in tools if isinstance(t, dict) and t]+ if tools:+ names = [t.get("function", {}).get("name", "") for t in tools]+ ids = [str(t.get("id", "")) for t in tools]+ args = [str(t.get("function", {}).get("arguments", "")) for t in tools]+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, ", ".join(filter(None, names))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, ", ".join(filter(None, ids))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS, ", ".join(filter(None, args))+ )+# Span Attributes for Cost and Tokensscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens@@ -534,87 +605,72 @@ def common_generate_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Tools- if scope._tools is not None:- if isinstance(scope._tools, dict):- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("function", "")- ).get("name", "")- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS,- str(scope._tools.get("function", "").get("arguments", "")),- )- elif isinstance(scope._tools, list) and len(scope._tools) > 0:- for tool in scope._tools:- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME,- tool.get("function", {}).get("name", ""),- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(tool.get("id", ""))- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS,- str(tool.get("function", {}).get("arguments", "")),- )+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )- # Span Attributes for Content+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ input_msgs = [+ {"role": "user", "parts": [{"type": "text", "content": prompt or ""}]}+ ]+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, scope._tools)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)- # Emit OTel log event+ # Emit inference eventif event_provider:try:- json_body = scope._kwargs.get("json", {}) or {}-- # For generate operation, input is a prompt string, not messages- input_msgs = [- {"role": "user", "parts": [{"type": "text", "content": prompt}]}- ]- output_msgs = build_output_messages(- scope._llmresponse, scope._finish_reason, scope._tools- )tool_defs = build_tool_definitions(json_body.get("tools", scope._kwargs.get("tools")))-- # Extract options for parameters- options = json_body.get("options", scope._kwargs.get("options", {}))-+ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": options.get("temperature"),+ "max_tokens": options.get("max_tokens"),+ "top_p": options.get("top_p"),+ "top_k": options.get("top_k"),+ "repeat_penalty": options.get("repeat_penalty"),+ "input_tokens": scope._input_tokens,+ "output_tokens": scope._output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_TEXT_COMPLETION,request_model=request_model,- response_model=scope._response_model,+ response_model=getattr(scope, "_response_model", request_model),input_messages=input_msgs,output_messages=output_msgs,tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- finish_reasons=[scope._finish_reason],- temperature=options.get("temperature"),- max_tokens=options.get("max_tokens"),- top_p=options.get("top_p"),- top_k=options.get("top_k"),- repeat_penalty=options.get("repeat_penalty"),- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,+ **extra,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_completion_metrics(metrics,@@ -623,14 +679,14 @@ def common_generate_logic(scope._server_address,scope._server_port,request_model,- scope._response_model,+ getattr(scope, "_response_model", request_model),environment,application_name,scope._start_time,scope._end_time,- cost,scope._input_tokens,scope._output_tokens,+ cost,scope._tbt,scope._ttft,)@@ -692,74 +748,32 @@ def common_embedding_logic(if capture_message_content:scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt_val)- # Emit OTel log event- if event_provider:- try:- # For embeddings, input is text strings, output is empty- if isinstance(prompt_val, str):- input_text = [prompt_val]- elif isinstance(prompt_val, list):- input_text = prompt_val- else:- input_text = [str(prompt_val)]-- # Create simple text input messages- input_msgs = [- {"role": "user", "parts": [{"type": "text", "content": text}]}- for text in input_text- ]-- emit_inference_event(- event_provider=event_provider,- operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_EMBEDDING,- request_model=request_model,- response_model=request_model,- input_messages=input_msgs,- output_messages=[], # Embeddings don't have text output- tool_definitions=None,- server_address=scope._server_address,- server_port=scope._server_port,- input_tokens=input_tokens,- )- except Exception as e:- import logging-- logger = logging.getLogger(__name__)- logger.warning("Failed to emit inference event: %s", e, exc_info=True)-- # Emit OTel log event+ # Emit inference eventif event_provider:try:- # For embeddings, input is text strings, output is emptyif isinstance(prompt_val, str):input_text = [prompt_val]elif isinstance(prompt_val, list):input_text = prompt_valelse:input_text = [str(prompt_val)]-- # Create simple text input messagesinput_msgs = [{"role": "user", "parts": [{"type": "text", "content": text}]}for text in input_text]-emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_EMBEDDING,request_model=request_model,response_model=request_model,input_messages=input_msgs,- output_messages=[], # Embeddings don't have text output+ output_messages=[],tool_definitions=None,server_address=scope._server_address,server_port=scope._server_port,input_tokens=input_tokens,)except Exception as e:- import logging-- logger = logging.getLogger(__name__)logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -800,7 +814,6 @@ def process_streaming_chat_response(common_chat_logic(self,- "ollama.chat",pricing_info,environment,application_name,@@ -808,13 +821,13 @@ def process_streaming_chat_response(capture_message_content,disable_metrics,version,- event_provider,+ is_stream=True,+ event_provider=event_provider,)def process_chat_response(response,- gen_ai_endpoint,pricing_info,server_port,server_address,@@ -841,8 +854,11 @@ def process_chat_response(scope._span = spanscope._llmresponse = response_dict.get("message", {}).get("content", "")scope._response_role = response_dict.get("message", {}).get("role", "assistant")+ # Handle token usage including reasoning tokens and cached tokensscope._input_tokens = response_dict.get("prompt_eval_count", 0)scope._output_tokens = response_dict.get("eval_count", 0)+ scope._cache_read_input_tokens = 0 # Ollama does not expose cached tokens+ scope._cache_creation_input_tokens = 0scope._response_model = response_dict.get("model", "llama3.2")scope._finish_reason = response_dict.get("done_reason", "")scope._timestamps = []@@ -852,13 +868,12 @@ def process_chat_response(scope._kwargs = kwargsif scope._kwargs.get("tools"):- scope._tools = response_dict.get("choices")[0].get("message").get("tool_calls")+ scope._tools = response_dict.get("message", {}).get("tool_calls")else:scope._tools = Nonecommon_chat_logic(scope,- gen_ai_endpoint,pricing_info,environment,application_name,@@ -866,7 +881,8 @@ def process_chat_response(capture_message_content,disable_metrics,version,- event_provider,+ is_stream=False,+ event_provider=event_provider,)return response@@ -889,7 +905,6 @@ def process_streaming_generate_response(common_generate_logic(self,- "ollama.generate",pricing_info,environment,application_name,@@ -897,13 +912,13 @@ def process_streaming_generate_response(capture_message_content,disable_metrics,version,- event_provider,+ is_stream=True,+ event_provider=event_provider,)def process_generate_response(response,- gen_ai_endpoint,pricing_info,server_port,server_address,@@ -930,8 +945,11 @@ def process_generate_response(scope._span = spanscope._llmresponse = response_dict.get("response", "")scope._response_role = response_dict.get("message", {}).get("role", "assistant")+ # Handle token usage including reasoning tokens and cached tokensscope._input_tokens = response_dict.get("prompt_eval_count", 0)scope._output_tokens = response_dict.get("eval_count", 0)+ scope._cache_read_input_tokens = 0 # Ollama does not expose cached tokens+ scope._cache_creation_input_tokens = 0scope._response_model = response_dict.get("model", "llama3.2")scope._finish_reason = response_dict.get("done_reason", "")scope._timestamps = []@@ -940,14 +958,11 @@ def process_generate_response(scope._server_address, scope._server_port = server_address, server_portscope._kwargs = kwargs- if scope._kwargs.get("tools"):- scope._tools = response_dict.get("choices")[0].get("message").get("tool_calls")- else:- scope._tools = None+ # Ollama generate response shape: no message.tool_calls in standard response+ scope._tools = Nonecommon_generate_logic(scope,- gen_ai_endpoint,pricing_info,environment,application_name,@@ -955,7 +970,8 @@ def process_generate_response(capture_message_content,disable_metrics,version,- event_provider,+ is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/openai/utils.py b/sdk/python/src/openlit/instrumentation/openai/utils.pyindex bb14d2d5..39da9303 100644--- a/sdk/python/src/openlit/instrumentation/openai/utils.py+++ b/sdk/python/src/openlit/instrumentation/openai/utils.py@@ -2,6 +2,7 @@OpenAI OpenTelemetry instrumentation utility functions"""+import jsonimport timeimport logging@@ -287,10 +288,8 @@ def build_output_messages(response_text, finish_reason, tool_calls=None):# Parse arguments if it's a stringif isinstance(tool_args, str):try:- import json-tool_args = json.loads(tool_args)- except:+ except Exception:tool_args = {"raw": tool_args}parts.append(@@ -390,6 +389,57 @@ def build_tool_definitions(tools):return None+def build_system_instructions_from_messages(messages):+ """+ Extract system message(s) from chat messages for gen_ai.system_instructions.+ Returns list of { type: "text", content: "..." } per OTel schema, or None.+ """+ if not messages:+ return None+ instructions = []+ for msg in messages:+ role = (+ msg.get("role", "") if isinstance(msg, dict) else getattr(msg, "role", "")+ )+ if role != "system":+ continue+ content = (+ msg.get("content", "")+ if isinstance(msg, dict)+ else getattr(msg, "content", "")+ )+ if isinstance(content, list):+ for part in content:+ if isinstance(part, dict) and part.get("type") == "text":+ text = part.get("text", "")+ if text:+ instructions.append({"type": "text", "content": text})+ elif content:+ instructions.append({"type": "text", "content": str(content)})+ return instructions if instructions else None+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)++def emit_inference_event(event_provider,operation_name,@@ -497,6 +547,10 @@ def emit_inference_event(attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = valueelse:# Pass through any other attributes as-isattributes[key] = value@@ -669,6 +723,12 @@ def process_response_chunk(scope, chunk):scope._response_id = response_data.get("id") or scope._response_idscope._response_model = response_data.get("model") or scope._response_modelscope._finish_reason = response_data.get("status")+ instructions_raw = response_data.get("instructions")+ scope._instructions = (+ [{"type": "text", "content": instructions_raw}]+ if instructions_raw+ else None+ )usage = response_data.get("usage", {})scope._input_tokens = usage.get("input_tokens", 0)@@ -678,6 +738,10 @@ def process_response_chunk(scope, chunk):output_tokens_details = usage.get("output_tokens_details", {})scope._reasoning_tokens = output_tokens_details.get("reasoning_tokens", 0)+ # Cached tokens (OTel: gen_ai.usage.cache_read.input_tokens)+ input_tokens_details = usage.get("input_tokens_details", {})+ scope._cache_read_input_tokens = input_tokens_details.get("cached_tokens", 0)+def common_response_logic(scope,@@ -731,6 +795,12 @@ def common_response_logic(version,)+ # OpenAI-specific API type attribute (TIER 2: OpenAI-Specific Semconv)+ scope._span.set_attribute(+ SemanticConvention.OPENAI_API_TYPE,+ "responses", # Responses API type+ )+# Span Attributes for Request parameters specific to responses APIscope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,@@ -753,10 +823,21 @@ def common_response_logic("gen_ai.request.reasoning_effort", reasoning.get("effort"))- # Responses API specific attributes- if hasattr(scope, "_service_tier"):+ # OpenAI: response.service_tier (from response); request.service_tier (from request when not 'auto')+ if hasattr(scope, "_service_tier") and scope._service_tier:+ scope._span.set_attribute(+ SemanticConvention.OPENAI_RESPONSE_SERVICE_TIER, scope._service_tier+ )+ req_tier = handle_not_given(scope._kwargs.get("service_tier"))+ if req_tier and str(req_tier) != "auto":+ scope._span.set_attribute(+ SemanticConvention.OPENAI_REQUEST_SERVICE_TIER, str(req_tier)+ )+ # gen_ai.request.choice.count when n != 1+ n_choices = handle_not_given(scope._kwargs.get("n"), 1)+ if n_choices != 1:scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_SERVICE_TIER, scope._service_tier+ SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT, int(n_choices))# Span Attributes for Response parameters@@ -821,25 +902,59 @@ def common_response_logic("gen_ai.usage.reasoning_tokens", scope._reasoning_tokens)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )++ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ input_msgs = build_input_messages(input_data)+ output_msgs = build_output_messages(+ scope._llmresponse,+ scope._finish_reason,+ scope._response_tools if hasattr(scope, "_response_tools") else None,)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ # gen_ai.system_instructions (Responses API: from response.instructions)+ if getattr(scope, "_instructions", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS,+ json.dumps(scope._instructions),+ )# Emit inference eventif event_provider:try:- input_msgs = build_input_messages(input_data)- output_msgs = build_output_messages(- scope._llmresponse,- scope._finish_reason,- scope._response_tools- if hasattr(scope, "_response_tools")- else None,- )-+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text",+ "temperature": handle_not_given(+ scope._kwargs.get("temperature"), 1.0+ ),+ "max_tokens": handle_not_given(+ scope._kwargs.get("max_output_tokens"), -1+ ),+ "top_p": handle_not_given(scope._kwargs.get("top_p"), 1.0),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ if getattr(scope, "_instructions", None):+ extra["system_instructions"] = scope._instructionsemit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,@@ -850,16 +965,7 @@ def common_response_logic(tool_definitions=None, # Responses API doesn't expose tools upfrontserver_address=scope._server_address,server_port=scope._server_port,- response_id=scope._response_id,- finish_reasons=[scope._finish_reason],- output_type="text",- temperature=handle_not_given(scope._kwargs.get("temperature"), 1.0),- max_tokens=handle_not_given(- scope._kwargs.get("max_output_tokens"), -1- ),- top_p=handle_not_given(scope._kwargs.get("top_p"), 1.0),- input_tokens=input_tokens,- output_tokens=output_tokens,+ **extra,)except Exception as e:logger.warning("Failed to emit inference event: %s", e, exc_info=True)@@ -979,7 +1085,7 @@ def process_response_response(scope._response_id = response_dict.get("id", "")scope._response_model = response_dict.get("model", "")- # Handle token usage including reasoning tokens+ # Handle token usage including reasoning tokens and cached tokensusage = response_dict.get("usage", {})scope._input_tokens = usage.get("input_tokens", 0)scope._output_tokens = usage.get("output_tokens", 0)@@ -987,12 +1093,20 @@ def process_response_response(output_tokens_details = usage.get("output_tokens_details", {})scope._reasoning_tokens = output_tokens_details.get("reasoning_tokens", 0)+ input_tokens_details = usage.get("input_tokens_details", {})+ scope._cache_read_input_tokens = input_tokens_details.get("cached_tokens", 0)+scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portscope._kwargs = kwargsscope._service_tier = response_dict.get("service_tier", "default")scope._finish_reason = response_dict.get("status", "completed")+ # Responses API: top-level instructions -> gen_ai.system_instructions+ instructions_raw = response_dict.get("instructions")+ scope._instructions = (+ [{"type": "text", "content": instructions_raw}] if instructions_raw else None+ )common_response_logic(scope,@@ -1068,10 +1182,17 @@ def common_chat_logic(version,)+ # OpenAI-specific API type attribute (TIER 2: OpenAI-Specific Semconv)+ scope._span.set_attribute(+ SemanticConvention.OPENAI_API_TYPE,+ "chat_completions", # Chat API type+ )+# Span Attributes for Request parameters+ seed_value = handle_not_given(scope._kwargs.get("seed"))scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED,- str(handle_not_given(scope._kwargs.get("seed"), "")),+ int(seed_value) if seed_value else 0,)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,@@ -1123,7 +1244,17 @@ def common_chat_logic()if hasattr(scope, "_service_tier") and scope._service_tier:scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_SERVICE_TIER, scope._service_tier+ SemanticConvention.OPENAI_RESPONSE_SERVICE_TIER, scope._service_tier+ )+ req_tier = handle_not_given(scope._kwargs.get("service_tier"))+ if req_tier and str(req_tier) != "auto":+ scope._span.set_attribute(+ SemanticConvention.OPENAI_REQUEST_SERVICE_TIER, str(req_tier)+ )+ n_choices = handle_not_given(scope._kwargs.get("n"), 1)+ if n_choices != 1:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT, int(n_choices))# Span Attributes for Tools - optimized@@ -1168,33 +1299,77 @@ def common_chat_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages / gen_ai.output.messages)+ if capture_message_content:+ if hasattr(scope, "_operation_type") and scope._operation_type == "responses":+ input_data = scope._kwargs.get("input", "")+ input_msgs = build_input_messages(input_data)+ else:+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse,+ scope._finish_reason,+ scope._tools if hasattr(scope, "_tools") else None,+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ # gen_ai.system_instructions (Chat: extract system messages from messages)+ chat_messages = scope._kwargs.get("messages", [])+ system_instr = build_system_instructions_from_messages(chat_messages)+ if system_instr:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS,+ json.dumps(system_instr),+ )+# Emit inference eventif event_provider:try:- # Get messages from kwargs (could be chat completions or responses API)- if (- hasattr(scope, "_operation_type")- and scope._operation_type == "responses"- ):- input_data = scope._kwargs.get("input", "")- input_msgs = build_input_messages(input_data)- else:- input_msgs = build_input_messages(scope._kwargs.get("messages", []))-- output_msgs = build_output_messages(- scope._llmresponse,- scope._finish_reason,- scope._tools if hasattr(scope, "_tools") else None,- )tool_defs = build_tool_definitions(scope._kwargs.get("tools"))-+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": handle_not_given(+ scope._kwargs.get("temperature"), 1.0+ ),+ "max_tokens": handle_not_given(scope._kwargs.get("max_tokens"), -1),+ "top_p": handle_not_given(scope._kwargs.get("top_p"), 1.0),+ "frequency_penalty": handle_not_given(+ scope._kwargs.get("frequency_penalty"), 0.0+ ),+ "presence_penalty": handle_not_given(+ scope._kwargs.get("presence_penalty"), 0.0+ ),+ "stop_sequences": handle_not_given(scope._kwargs.get("stop"), []),+ "seed": int(handle_not_given(scope._kwargs.get("seed"), 0))+ if handle_not_given(scope._kwargs.get("seed"))+ else None,+ "choice_count": scope._kwargs.get("n", 1),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ if system_instr:+ extra["system_instructions"] = system_instremit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,@@ -1205,29 +1380,7 @@ def common_chat_logic(tool_definitions=tool_defs,server_address=scope._server_address,server_port=scope._server_port,- response_id=scope._response_id,- finish_reasons=[scope._finish_reason],- output_type="text"- if isinstance(scope._llmresponse, str)- else "json",- temperature=handle_not_given(scope._kwargs.get("temperature"), 1.0),- max_tokens=handle_not_given(scope._kwargs.get("max_tokens"), -1),- top_p=handle_not_given(scope._kwargs.get("top_p"), 1.0),- frequency_penalty=handle_not_given(- scope._kwargs.get("frequency_penalty"), 0.0- ),- presence_penalty=handle_not_given(- scope._kwargs.get("presence_penalty"), 0.0- ),- stop_sequences=handle_not_given(scope._kwargs.get("stop"), []),- seed=str(handle_not_given(scope._kwargs.get("seed"), "")),- choice_count=scope._kwargs.get("n", 1),- input_tokens=input_tokens,- output_tokens=output_tokens,- cache_read_input_tokens=scope._cache_read_input_tokens- if hasattr(scope, "_cache_read_input_tokens")- and scope._cache_read_input_tokens > 0- else None,+ **extra,)except Exception as e:logger.warning("Failed to emit inference event: %s", e, exc_info=True)@@ -1412,6 +1565,17 @@ def common_embedding_logic(SemanticConvention.GEN_AI_REQUEST_USER,handle_not_given(scope._kwargs.get("user"), ""),)+ # gen_ai.embeddings.dimension.count (from request or response when available)+ dim_count = handle_not_given(scope._kwargs.get("dimensions"))+ if dim_count is None and getattr(scope, "_embedding_dimension", None) is not None:+ dim_count = scope._embedding_dimension+ if dim_count is not None:+ try:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_EMBEDDINGS_DIMENSION_COUNT, int(dim_count)+ )+ except (TypeError, ValueError):+ pass# Span Attributes for Cost and Tokensscope._span.set_attribute(@@ -1422,38 +1586,26 @@ def common_embedding_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content+ # Span Attributes for Content (OTel: array structure for gen_ai.input.messages)if capture_message_content:input_data = scope._kwargs.get("input", "")- formatted_content = (- format_content(input_data)- if isinstance(input_data, (list, dict))- else str(input_data)- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_INPUT_MESSAGES, formatted_content- )+ if isinstance(input_data, list):+ input_msgs = [+ {"role": "user", "parts": [{"type": "text", "content": str(item)}]}+ for item in input_data+ ]+ else:+ input_msgs = [+ {+ "role": "user",+ "parts": [{"type": "text", "content": str(input_data)}],+ }+ ]+ _set_span_messages_as_array(scope._span, input_msgs, None)# Emit inference eventif event_provider:try:- # Build input messages with text content- if isinstance(input_data, list):- input_msgs = [- {- "role": "user",- "parts": [{"type": "text", "content": str(item)}],- }- for item in input_data- ]- else:- input_msgs = [- {- "role": "user",- "parts": [{"type": "text", "content": str(input_data)}],- }- ]-emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_EMBEDDING,@@ -1592,11 +1744,15 @@ def common_image_logic(# Span Attributes for Costscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content+ # Span Attributes for Content (OTel: array structure)if capture_message_content:- # Always collect the original promptprompt = scope._kwargs.get("prompt", "")- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ input_msgs = (+ [{"role": "user", "parts": [{"type": "text", "content": prompt}]}]+ if prompt+ else []+ )+ output_msgs = []# Collect and set revised prompts if availableif images_data:@@ -1611,45 +1767,33 @@ def common_image_logic(SemanticConvention.GEN_AI_CONTENT_REVISED_PROMPT, revised_prompts)- # Emit inference event+ # Build output messages with revised prompts as text parts+ for image in images_data:+ parts = []+ if image.get("revised_prompt"):+ parts.append({"type": "text", "content": image["revised_prompt"]})+ if image.get("url"):+ parts.append(+ {+ "type": "uri",+ "modality": "image",+ "uri": image["url"],+ }+ )+ if parts:+ output_msgs.append(+ {+ "role": "assistant",+ "parts": parts,+ "finish_reason": "stop",+ }+ )+ _set_span_messages_as_array(+ scope._span, input_msgs, output_msgs if output_msgs else None+ )+if event_provider:try:- # Build input messages with text prompt- input_msgs = (- [{"role": "user", "parts": [{"type": "text", "content": prompt}]}]- if prompt- else []- )-- # Build output messages with revised prompts as text parts- output_msgs = []- if images_data:- for image in images_data:- parts = []- # Add revised prompt if available- if image.get("revised_prompt"):- parts.append(- {"type": "text", "content": image["revised_prompt"]}- )- # Add image URL if available (not base64)- if image.get("url"):- parts.append(- {- "type": "uri",- "modality": "image",- "uri": image["url"],- }- )-- if parts:- output_msgs.append(- {- "role": "assistant",- "parts": parts,- "finish_reason": "stop",- }- )-emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_IMAGE,@@ -1741,35 +1885,25 @@ def common_audio_logic(# Span Attributes for Costscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content+ # Span Attributes for Content (OTel: array structure)if capture_message_content:input_text = scope._kwargs.get("input", "")- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, input_text)+ input_msgs = (+ [{"role": "user", "parts": [{"type": "text", "content": input_text}]}]+ if input_text+ else []+ )+ output_msgs = [+ {+ "role": "assistant",+ "parts": [{"type": "text", "content": "[audio generated]"}],+ "finish_reason": "stop",+ }+ ]+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)- # Emit inference eventif event_provider:try:- # Build input messages with text content- input_msgs = (- [- {- "role": "user",- "parts": [{"type": "text", "content": input_text}],- }- ]- if input_text- else []- )-- # Audio output doesn't have text response, just indicate audio generation- output_msgs = [- {- "role": "assistant",- "parts": [{"type": "text", "content": "[audio generated]"}],- "finish_reason": "stop",- }- ]-emit_inference_event(event_provider=event_provider,operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_AUDIO,@@ -1884,6 +2018,12 @@ def process_embedding_response(scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portscope._kwargs = kwargs+ # gen_ai.embeddings.dimension.count from response when not in request+ data = response_dict.get("data", [])+ if data and isinstance(data[0], dict) and data[0].get("embedding"):+ scope._embedding_dimension = len(data[0]["embedding"])+ else:+ scope._embedding_dimension = Nonecommon_embedding_logic(scope,diff --git a/sdk/python/src/openlit/instrumentation/premai/__init__.py b/sdk/python/src/openlit/instrumentation/premai/__init__.pyindex 21d20df3..09b1556d 100644--- a/sdk/python/src/openlit/instrumentation/premai/__init__.py+++ b/sdk/python/src/openlit/instrumentation/premai/__init__.py@@ -26,6 +26,7 @@ class PremAIInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("premai")# sync chat@@ -41,6 +42,7 @@ class PremAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/premai/premai.py b/sdk/python/src/openlit/instrumentation/premai/premai.pyindex 8390ce74..6743fc63 100644--- a/sdk/python/src/openlit/instrumentation/premai/premai.py+++ b/sdk/python/src/openlit/instrumentation/premai/premai.py@@ -23,6 +23,7 @@ def chat(capture_message_content,metrics,disable_metrics,+ event_provider,):"""Generates a telemetry wrapper for GenAI function call@@ -51,6 +52,8 @@ def chat(self._response_model = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._finish_reason = ""self._tools = Noneself._args = args@@ -93,6 +96,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:@@ -142,6 +146,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/premai/utils.py b/sdk/python/src/openlit/instrumentation/premai/utils.pyindex 532acda8..5cb18788 100644--- a/sdk/python/src/openlit/instrumentation/premai/utils.py+++ b/sdk/python/src/openlit/instrumentation/premai/utils.py@@ -2,6 +2,8 @@PremAI OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -14,11 +16,14 @@ from openlit.__helpers import (get_embed_model_cost,general_tokens,common_span_attributes,+ otel_event,record_completion_metrics,record_embedding_metrics,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -44,32 +49,262 @@ def format_content(messages):return "\n".join(formatted_messages)+def build_input_messages(messages):+ """+ Convert Premai request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not messages:+ return []+ if isinstance(messages, str):+ return [{"role": "user", "parts": [{"type": "text", "content": messages}]}]+ otel_messages = []+ for message in messages:+ try:+ role = message.get("role", "user")+ content = message.get("content", "")+ parts = []+ if isinstance(content, list):+ for item in content:+ if not isinstance(item, dict):+ continue+ item_type = item.get("type")+ if item_type == "text":+ text_content = item.get("text", "")+ if text_content:+ parts.append({"type": "text", "content": text_content})+ elif item_type == "image_url":+ image_url_obj = item.get("image_url", {})+ url = (+ image_url_obj.get("url", "")+ if isinstance(image_url_obj, dict)+ else ""+ )+ if url and not url.startswith("data:"):+ parts.append(+ {"type": "uri", "modality": "image", "uri": url}+ )+ elif isinstance(content, str) and content:+ parts.append({"type": "text", "content": content})+ if parts:+ otel_messages.append({"role": role, "parts": parts})+ except Exception as e:+ logger.warning("Failed to process input message: %s", e, exc_info=True)+ continue+ return otel_messages+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Premai response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ try:+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls and isinstance(tool_calls, list):+ for tool_call in tool_calls:+ try:+ if isinstance(tool_call, dict):+ func = tool_call.get("function", {})+ tool_id = tool_call.get("id", "")+ tool_name = func.get("name", "")+ tool_args = func.get("arguments", {})+ else:+ tool_id = getattr(tool_call, "id", "")+ func = getattr(tool_call, "function", {}) or {}+ tool_name = (+ func.get("name", "")+ if isinstance(func, dict)+ else getattr(func, "name", "")+ )+ tool_args = (+ func.get("arguments", "")+ if isinstance(func, dict)+ else getattr(func, "arguments", "")+ )+ if isinstance(tool_args, str):+ try:+ tool_args = json.loads(tool_args)+ except Exception:+ tool_args = {"raw": tool_args}+ parts.append(+ {+ "type": "tool_call",+ "id": tool_id,+ "name": tool_name,+ "arguments": tool_args,+ }+ )+ except Exception as e:+ logger.warning("Failed to process tool call: %s", e, exc_info=True)+ finish_reason_map = {+ "stop": "stop",+ "length": "length",+ "content_filter": "content_filter",+ "tool_calls": "tool_call",+ }+ otel_finish_reason = finish_reason_map.get(+ finish_reason, finish_reason or "stop"+ )+ return [+ {"role": "assistant", "parts": parts, "finish_reason": otel_finish_reason}+ ]+ except Exception as e:+ logger.warning("Failed to build output messages: %s", e, exc_info=True)+ return [{"role": "assistant", "parts": [], "finish_reason": "stop"}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def process_chunk(scope, chunk):"""Process a chunk of response data and update state."""end_time = time.time()- # Record the timestamp for the current chunkscope._timestamps.append(end_time)if len(scope._timestamps) == 1:- # Calculate time to first chunkscope._ttft = calculate_ttft(scope._timestamps, scope._start_time)chunked = response_as_dict(chunk)- # Collect message IDs and aggregated response from events- if chunked.choices:- first_choice = chunked.get("choices")[0]-- if first_choice.get("delta").get("content"):- scope._llmresponse += first_choice.get("delta").get("content")-- if chunked.get("choices")[0].get("finish_reason"):- scope._finish_reason = chunked.get("choices")[0].get("finish_reason")- scope._response_id = chunked.get("id")- scope._response_model = chunked.get("model")- scope._end_time = time.time()+ choices = chunked.get("choices", []) if isinstance(chunked, dict) else []+ if choices:+ first_choice = choices[0] if isinstance(choices[0], dict) else {}+ delta = first_choice.get("delta") or {}+ if isinstance(delta, dict) and delta.get("content"):+ scope._llmresponse += delta.get("content", "")+ if first_choice.get("finish_reason"):+ scope._finish_reason = first_choice.get("finish_reason")+ scope._response_id = chunked.get("id")+ scope._response_model = chunked.get("model")+ scope._end_time = time.time()++ usage = chunked.get("usage") if isinstance(chunked, dict) else None+ if usage and isinstance(usage, dict):+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)+ input_tokens_details = usage.get("input_tokens_details") or usage.get(+ "prompt_tokens_details", {}+ )+ scope._cache_read_input_tokens = input_tokens_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = usage.get("cache_creation_input_tokens", 0)def common_chat_logic(@@ -82,18 +317,20 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""- Process chat request and generate Telemetry+ Process chat request and generate Telemetry."""+ scope._end_time = time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)prompt = format_content(scope._kwargs.get("messages", ""))request_model = scope._kwargs.get("model", "llama3.2-3b")- # Calculate tokens using input prompt and aggregated response+ # Compute costif is_stream:input_tokens = general_tokens(prompt)output_tokens = general_tokens(scope._llmresponse)@@ -120,10 +357,9 @@ def common_chat_logic(version,)- # Span Attributes for Response parameters- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason]- )+ # Premai chat (no additional API type attribute)++ # Span Attributes for Request parametersscope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED, scope._kwargs.get("seed", ""))@@ -140,17 +376,26 @@ def common_chat_logic(scope._kwargs.get("presence_penalty", 0.0),)scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, scope._kwargs.get("stop", [])+ SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,+ scope._kwargs.get("stop", []),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,scope._kwargs.get("temperature", 1.0),)scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_TOP_P, scope._kwargs.get("top_p", 1.0)+ SemanticConvention.GEN_AI_REQUEST_TOP_P,+ scope._kwargs.get("top_p", 1.0),)scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_USER, scope._kwargs.get("user", "")+ SemanticConvention.GEN_AI_REQUEST_USER,+ scope._kwargs.get("user", ""),+ )++ # Span Attributes for Response parameters+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [scope._finish_reason],)scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)scope._span.set_attribute(@@ -158,6 +403,43 @@ def common_chat_logic("text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if scope._tools:+ tools = scope._tools if isinstance(scope._tools, list) else [scope._tools]+ names, ids, args = (+ zip(+ *[+ (+ t.get("function", {}).get("name", "")+ if isinstance(t, dict)+ else "",+ str(t.get("id", "")) if isinstance(t, dict) else "",+ str(+ t.get("function", {}).get("arguments", "")+ if isinstance(t, dict)+ else ""+ ),+ )+ for t in tools+ if isinstance(t, dict) and t+ ]+ )+ if tools+ else ([], [], [])+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME,+ ", ".join(filter(None, names)),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID,+ ", ".join(filter(None, ids)),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS,+ ", ".join(filter(None, args)),+ )+# Span Attributes for Cost and Tokensscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens@@ -166,47 +448,72 @@ def common_chat_logic(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Tools- if scope._tools:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("function", "")- ).get("name", "")- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS,- str(scope._tools.get("function", "").get("arguments", "")),+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)# Span Attributes for Contentif capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, scope._tools)-- # To be removed one the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },- )- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },- )-+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ extra = {+ "response_id": scope._response_id,+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature", 1.0),+ "max_tokens": scope._kwargs.get("max_tokens", -1),+ "top_p": scope._kwargs.get("top_p", 1.0),+ "frequency_penalty": scope._kwargs.get("frequency_penalty", 0.0),+ "presence_penalty": scope._kwargs.get("presence_penalty", 0.0),+ "stop_sequences": scope._kwargs.get("stop", []),+ "seed": scope._kwargs.get("seed", ""),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=scope._response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++ # Span status and metricsscope._span.set_status(Status(StatusCode.OK))-- # Metricsif not disable_metrics:record_completion_metrics(metrics,@@ -320,6 +627,7 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process chat request and generate Telemetry@@ -334,6 +642,7 @@ def process_streaming_chat_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -351,6 +660,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -359,28 +669,37 @@ def process_chat_response(scope = type("GenericScope", (), {})()response_dict = response_as_dict(response)+ choices = response_dict.get("choices", [])+ first_choice = choices[0] if choices else {}+ message = first_choice.get("message") or {}+ usage = response_dict.get("usage") or {}scope._start_time = start_timescope._end_time = time.time()scope._span = span- scope._llmresponse = str(- response_dict.get("choices")[0].get("message").get("content")- )+ scope._llmresponse = str(message.get("content", ""))scope._response_id = response_dict.get("additional_properties", {}).get("id")scope._response_model = response_dict.get("model")- scope._input_tokens = response_dict.get("usage").get("prompt_tokens")- scope._output_tokens = response_dict.get("usage").get("completion_tokens")+ scope._input_tokens = usage.get("prompt_tokens", 0)+ scope._output_tokens = usage.get("completion_tokens", 0)scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portscope._kwargs = kwargs- scope._finish_reason = str(response_dict.get("choices")[0].get("finish_reason"))+ scope._finish_reason = str(first_choice.get("finish_reason", ""))if scope._kwargs.get("tools"):- scope._tools = response_dict.get("choices")[0].get("message").get("tool_calls")+ scope._tools = message.get("tool_calls")else:scope._tools = None+ # Handle token usage including reasoning tokens and cached tokens+ input_tokens_details = usage.get("input_tokens_details") or usage.get(+ "prompt_tokens_details", {}+ )+ scope._cache_read_input_tokens = input_tokens_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = usage.get("cache_creation_input_tokens", 0)+common_chat_logic(scope,pricing_info,@@ -391,6 +710,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/reka/__init__.py b/sdk/python/src/openlit/instrumentation/reka/__init__.pyindex 97ad070a..7a112714 100644--- a/sdk/python/src/openlit/instrumentation/reka/__init__.py+++ b/sdk/python/src/openlit/instrumentation/reka/__init__.py@@ -27,6 +27,7 @@ class RekaInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("reka-api")# Chat completions@@ -42,6 +43,7 @@ class RekaInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -58,6 +60,7 @@ class RekaInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/reka/async_reka.py b/sdk/python/src/openlit/instrumentation/reka/async_reka.pyindex b37c06ff..b6e07994 100644--- a/sdk/python/src/openlit/instrumentation/reka/async_reka.py+++ b/sdk/python/src/openlit/instrumentation/reka/async_reka.py@@ -18,6 +18,7 @@ def async_chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -54,6 +55,7 @@ def async_chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/reka/reka.py b/sdk/python/src/openlit/instrumentation/reka/reka.pyindex 7853b7d4..1670a181 100644--- a/sdk/python/src/openlit/instrumentation/reka/reka.py+++ b/sdk/python/src/openlit/instrumentation/reka/reka.py@@ -18,6 +18,7 @@ def chat(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -54,6 +55,7 @@ def chat(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/reka/utils.py b/sdk/python/src/openlit/instrumentation/reka/utils.pyindex 5ff4658b..dc4f2e8b 100644--- a/sdk/python/src/openlit/instrumentation/reka/utils.py+++ b/sdk/python/src/openlit/instrumentation/reka/utils.py@@ -2,6 +2,8 @@Reka OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -9,12 +11,16 @@ from opentelemetry.trace import Status, StatusCodefrom openlit.__helpers import (response_as_dict,calculate_tbt,+ general_tokens,get_chat_model_cost,common_span_attributes,record_completion_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -40,6 +46,215 @@ def format_content(messages):return "\n".join(formatted_messages)+def build_input_messages(messages):+ """+ Convert Reka request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not messages:+ return []+ otel_messages = []+ for message in messages:+ try:+ role = (+ message.get("role", "user")+ if isinstance(message, dict)+ else getattr(message, "role", "user")+ )+ content = (+ message.get("content", "")+ if isinstance(message, dict)+ else getattr(message, "content", "")+ )+ parts = []+ if isinstance(content, list):+ for item in content:+ if isinstance(item, dict) and item.get("type") == "text":+ text = item.get("text", "")+ if text:+ parts.append({"type": "text", "content": text})+ elif isinstance(item, dict) and item.get("type") == "image_url":+ url = (item.get("image_url") or {}).get("url", "")+ if url and not str(url).startswith("data:"):+ parts.append(+ {"type": "uri", "modality": "image", "uri": url}+ )+ elif isinstance(content, str) and content:+ parts.append({"type": "text", "content": content})+ if role == "tool":+ tool_call_id = (+ message.get("tool_call_id", "")+ if isinstance(message, dict)+ else getattr(message, "tool_call_id", "")+ )+ tool_content = content if isinstance(content, str) else str(content)+ parts = [+ {+ "type": "tool_call_response",+ "id": tool_call_id,+ "response": tool_content,+ }+ ]+ if parts:+ otel_messages.append({"role": role, "parts": parts})+ except Exception as e:+ logger.warning("Failed to process input message: %s", e, exc_info=True)+ continue+ return otel_messages+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Reka response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ try:+ if response_text:+ parts.append({"type": "text", "content": response_text})+ if tool_calls:+ tools = tool_calls if isinstance(tool_calls, list) else [tool_calls]+ for t in tools:+ if isinstance(t, dict):+ # Reka shape: name, id, parameters+ params = t.get("parameters", "")+ if isinstance(params, str):+ try:+ params = json.loads(params) if params else {}+ except Exception:+ params = {"raw": params}+ parts.append(+ {+ "type": "tool_call",+ "id": t.get("id", ""),+ "name": t.get("name", ""),+ "arguments": params,+ }+ )+ finish_reason_map = {+ "stop": "stop",+ "length": "length",+ "content_filter": "content_filter",+ "tool_calls": "tool_call",+ "function_call": "tool_call",+ }+ otel_finish = finish_reason_map.get(finish_reason, finish_reason or "stop")+ return [{"role": "assistant", "parts": parts, "finish_reason": otel_finish}]+ except Exception as e:+ logger.warning("Failed to build output messages: %s", e, exc_info=True)+ return [{"role": "assistant", "parts": [], "finish_reason": "stop"}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port is not None:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def common_chat_logic(scope,pricing_info,@@ -50,20 +265,32 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process chat request and generate Telemetry"""+ scope._end_time = time.time()if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- prompt = format_content(scope._kwargs.get("messages", ""))+ prompt = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "reka-core-20240501")- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ if (+ hasattr(scope, "_input_tokens")+ and scope._input_tokens is not None+ and hasattr(scope, "_output_tokens")+ and scope._output_tokens is not None+ ):+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)++ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributescommon_span_attributes(@@ -73,7 +300,7 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- scope._response_model,+ getattr(scope, "_response_model", request_model),environment,application_name,is_stream,@@ -82,10 +309,7 @@ def common_chat_logic(version,)- # Span Attributes for Response parameters- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason]- )+ # Span Attributes for Request parametersscope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED, scope._kwargs.get("seed", ""))@@ -110,57 +334,103 @@ def common_chat_logic(scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, scope._kwargs.get("top_p", 1.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)++ # Span Attributes for Response parameters+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [scope._finish_reason]+ )scope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE,"text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if scope._tools:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("name", "")+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS,+ str(scope._tools.get("parameters", "")),+ )+# Span Attributes for Cost and Tokensscope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Tools- if scope._tools:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_NAME, scope._tools.get("name", "")- )- scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_CALL_ID, str(scope._tools.get("id", ""))+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_TOOL_ARGS, str(scope._tools.get("parameters", ""))+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)# Span Attributes for Contentif capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse, scope._finish_reason, tool_calls=scope._tools)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)- # To be removed one the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },- )- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },- )+ if event_provider:+ try:+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [scope._finish_reason],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop"),+ "seed": scope._kwargs.get("seed"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=request_model,+ response_model=getattr(scope, "_response_model", request_model),+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -173,13 +443,13 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- scope._response_model,+ getattr(scope, "_response_model", request_model),environment,application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -200,6 +470,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -209,26 +480,39 @@ def process_chat_response(scope = type("GenericScope", (), {})()response_dict = response_as_dict(response)+ responses_list = response_dict.get("responses") or []+ first_response = responses_list[0] if responses_list else {}+ first_message = (+ first_response.get("message", {}) if isinstance(first_response, dict) else {}+ )+scope._start_time = start_timescope._end_time = time.time()scope._span = spanscope._llmresponse = " ".join(- (choice.get("message", {}).get("content") or "")- for choice in response_dict.get("responses", [])+ (r.get("message", {}).get("content") or "") for r in responses_list)scope._response_id = response_dict.get("id")- scope._response_model = response_dict.get("model")- scope._input_tokens = response_dict.get("usage").get("input_tokens")- scope._output_tokens = response_dict.get("usage").get("output_tokens")+ scope._response_model = response_dict.get("model") or request_model+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage") or {}+ scope._input_tokens = usage.get("input_tokens", 0)+ scope._output_tokens = usage.get("output_tokens") or usage.get(+ "generated_tokens", 0+ )+ scope._cache_read_input_tokens = (usage.get("prompt_tokens_details") or {}).get(+ "cached_tokens", 0+ ) or 0+ scope._cache_creation_input_tokens = (usage.get("input_tokens_details") or {}).get(+ "cache_creation_tokens", 0+ ) or 0scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portscope._kwargs = kwargs- scope._finish_reason = str(response_dict.get("responses")[0].get("finish_reason"))+ scope._finish_reason = str(first_response.get("finish_reason", ""))if scope._kwargs.get("tools"):- scope._tools = (- response_dict.get("responses")[0].get("message").get("tool_calls")- )+ scope._tools = first_message.get("tool_calls")else:scope._tools = None@@ -242,6 +526,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/sarvam/__init__.py b/sdk/python/src/openlit/instrumentation/sarvam/__init__.pyindex 2b88d950..6928ec1e 100644--- a/sdk/python/src/openlit/instrumentation/sarvam/__init__.py+++ b/sdk/python/src/openlit/instrumentation/sarvam/__init__.py@@ -44,6 +44,7 @@ class SarvamInstrumentor(BaseInstrumentor):capture_message_content = kwargs.get("capture_message_content", False)metrics = kwargs.get("metrics_dict")disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")# Chat completions - synctry:@@ -59,6 +60,7 @@ class SarvamInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider=event_provider,),)except Exception:@@ -78,6 +80,7 @@ class SarvamInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider=event_provider,),)except Exception:diff --git a/sdk/python/src/openlit/instrumentation/sarvam/async_sarvam.py b/sdk/python/src/openlit/instrumentation/sarvam/async_sarvam.pyindex e55acc09..7dedc9da 100644--- a/sdk/python/src/openlit/instrumentation/sarvam/async_sarvam.py+++ b/sdk/python/src/openlit/instrumentation/sarvam/async_sarvam.py@@ -29,6 +29,7 @@ def async_chat_completions(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for async Sarvam AI Chat Completions calls.@@ -47,6 +48,7 @@ def async_chat_completions(kwargs,server_address,server_port,+ event_provider=None,):self.__wrapped__ = wrappedself._span = span@@ -66,6 +68,9 @@ def async_chat_completions(self._tbt = 0self._server_address = server_addressself._server_port = server_port+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0+ self._event_provider = event_providerasync def __aenter__(self):await self.__wrapped__.__aenter__()@@ -98,6 +103,7 @@ def async_chat_completions(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=self._event_provider,)except Exception as e:handle_exception(self._span, e)@@ -126,7 +132,13 @@ def async_chat_completions(span = tracer.start_span(span_name, kind=SpanKind.CLIENT)return TracedAsyncStream(- awaited_wrapped, span, span_name, kwargs, server_address, server_port+ awaited_wrapped,+ span,+ span_name,+ kwargs,+ server_address,+ server_port,+ event_provider=event_provider,)else:@@ -149,6 +161,7 @@ def async_chat_completions(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/sarvam/sarvam.py b/sdk/python/src/openlit/instrumentation/sarvam/sarvam.pyindex 98c6c626..ca5eb550 100644--- a/sdk/python/src/openlit/instrumentation/sarvam/sarvam.py+++ b/sdk/python/src/openlit/instrumentation/sarvam/sarvam.py@@ -31,6 +31,7 @@ def chat_completions(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for Sarvam AI Chat completions calls.@@ -49,6 +50,7 @@ def chat_completions(kwargs,server_address,server_port,+ event_provider=None,):self.__wrapped__ = wrappedself._span = span@@ -68,6 +70,9 @@ def chat_completions(self._tbt = 0self._server_address = server_addressself._server_port = server_port+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0+ self._event_provider = event_providerdef __enter__(self):self.__wrapped__.__enter__()@@ -100,6 +105,7 @@ def chat_completions(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=self._event_provider,)except Exception as e:handle_exception(self._span, e)@@ -128,7 +134,13 @@ def chat_completions(span = tracer.start_span(span_name, kind=SpanKind.CLIENT)return TracedSyncStream(- awaited_wrapped, span, span_name, kwargs, server_address, server_port+ awaited_wrapped,+ span,+ span_name,+ kwargs,+ server_address,+ server_port,+ event_provider=event_provider,)else:@@ -151,6 +163,7 @@ def chat_completions(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/sarvam/utils.py b/sdk/python/src/openlit/instrumentation/sarvam/utils.pyindex 26850db5..869797f5 100644--- a/sdk/python/src/openlit/instrumentation/sarvam/utils.py+++ b/sdk/python/src/openlit/instrumentation/sarvam/utils.py@@ -2,6 +2,8 @@Sarvam AI OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -12,12 +14,16 @@ from openlit.__helpers import (calculate_tbt,get_chat_model_cost,record_completion_metrics,+ common_span_attributes,common_framework_span_attributes,+ otel_event,get_audio_model_cost,record_audio_metrics,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(messages):"""@@ -37,6 +43,161 @@ def format_content(messages):return "\n".join(formatted_messages)+def build_input_messages(messages):+ """+ Convert Sarvam request messages to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not messages:+ return []+ result = []+ for msg in messages:+ role = msg.get("role", "user")+ content = msg.get("content", "")+ if isinstance(content, list):+ parts = [+ {+ "type": p.get("type", "text"),+ "content": p.get("text", p.get("content", "")),+ }+ for p in content+ ]+ else:+ parts = [{"type": "text", "content": str(content) if content else ""}]+ result.append({"role": role, "parts": parts})+ return result+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Sarvam response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls:+ tools = tool_calls if isinstance(tool_calls, list) else [tool_calls]+ for t in tools:+ tc = t if isinstance(t, dict) else {}+ fn = tc.get("function", {}) or {}+ parts.append(+ {+ "type": "tool_call",+ "id": tc.get("id", ""),+ "name": fn.get("name", ""),+ "arguments": fn.get("arguments", ""),+ }+ )+ reason = finish_reason if finish_reason else "stop"+ return [{"role": "assistant", "parts": parts, "finish_reason": reason}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type" and value is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n") and value is not None and value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value if value is not None else 0+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value if value is not None else 0+ elif key == "error_type" and value is not None:+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions" and value is not None:+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "conversation_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def process_chunk(scope, chunk):"""Processes a streaming chunk from Sarvam AI API response.@@ -73,6 +234,14 @@ def process_chunk(scope, chunk):scope._input_tokens = chunk.usage.prompt_tokensif hasattr(chunk.usage, "completion_tokens"):scope._output_tokens = chunk.usage.completion_tokens+ if hasattr(scope, "_cache_read_input_tokens"):+ pt_details = getattr(chunk.usage, "prompt_tokens_details", None)+ if pt_details and getattr(pt_details, "cached_tokens", None) is not None:+ scope._cache_read_input_tokens = pt_details.cached_tokens+ if hasattr(scope, "_cache_creation_input_tokens"):+ cache_creation = getattr(chunk.usage, "cache_creation_input_tokens", None)+ if cache_creation is not None:+ scope._cache_creation_input_tokens = cache_creation# Extract response ID and model if availableif hasattr(chunk, "id"):@@ -91,168 +260,220 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""- Common logic for processing chat responses (both streaming and non-streaming).-- Args:- scope: The scope object containing span and response data.- pricing_info: Information about model pricing.- environment: The environment name.- application_name: The application name.- metrics: Metrics collection object.- capture_message_content: Whether to capture message content.- disable_metrics: Whether to disable metrics collection.- version: The version of the instrumentation.- is_stream: Whether this is a streaming response.+ Process chat request and generate Telemetry."""scope._end_time = time.time()- # Calculate time between tokens for streaming responsesif len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- # Format messages for capture- formatted_messages = format_content(scope._kwargs.get("messages", []))request_model = scope._kwargs.get("model", "sarvam-m")+ input_tokens = getattr(scope, "_input_tokens", 0)+ output_tokens = getattr(scope, "_output_tokens", 0)- # Calculate cost- cost = get_chat_model_cost(- request_model, pricing_info, scope._input_tokens, scope._output_tokens- )+ # Compute cost+ cost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)- # Common Framework Span Attributes- common_framework_span_attributes(- scope=scope,- framework_system=SemanticConvention.GEN_AI_SYSTEM_SARVAM,- server_address=scope._server_address,- server_port=scope._server_port,- environment=environment,- application_name=application_name,- version=version,- endpoint=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- )-- # Additional GenAI-specific attributes- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, request_model)- scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_MODEL, scope._response_model+ # Common Span Attributes+ common_span_attributes(+ scope,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_SARVAM,+ scope._server_address,+ scope._server_port,+ request_model,+ getattr(scope, "_response_model", request_model),+ environment,+ application_name,+ is_stream,+ scope._tbt,+ scope._ttft,+ version,)- if is_stream:- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, True)-- if scope._tbt > 0:- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TBT, scope._tbt)-- if scope._ttft > 0:- scope._span.set_attribute(SemanticConvention.GEN_AI_SERVER_TTFT, scope._ttft)+ # Sarvam-specific parameters+ if scope._kwargs.get("reasoning_effort"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_REASONING_EFFORT,+ scope._kwargs.get("reasoning_effort"),+ )- # Span Attributes for Request parameters (using Sarvam API defaults)+ # Span Attributes for Request parametersscope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,scope._kwargs.get("max_tokens", -1),)scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, scope._kwargs.get("stop", [])+ SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES,+ scope._kwargs.get("stop", []),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,- scope._kwargs.get("temperature", 0.2), # API default+ scope._kwargs.get("temperature", 0.2),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TOP_P,- scope._kwargs.get("top_p", 1.0), # API default+ scope._kwargs.get("top_p", 1.0),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,- scope._kwargs.get("frequency_penalty", 0.0), # API default+ scope._kwargs.get("frequency_penalty", 0.0),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,- scope._kwargs.get("presence_penalty", 0.0), # API default- )-- # Additional chat parameters with API defaults- scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_N,- scope._kwargs.get("n", 1), # API default+ scope._kwargs.get("presence_penalty", 0.0),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM,- scope._kwargs.get("stream", False), # API default+ scope._kwargs.get("stream", False),)scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_WIKI_GROUNDING,- scope._kwargs.get("wiki_grounding", False), # API default+ scope._kwargs.get("wiki_grounding", False),)-if scope._kwargs.get("seed") is not None:scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED, scope._kwargs.get("seed"))-- # Sarvam-specific parameters- if scope._kwargs.get("reasoning_effort"):+ n_choices = scope._kwargs.get("n", 1)+ if n_choices != 1:scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_REASONING_EFFORT,- scope._kwargs.get("reasoning_effort"),+ SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT, int(n_choices))- # Span Attributes for Response- scope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ # Span Attributes for Response parameters+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )+ finish_reason = getattr(scope, "_finish_reason", None) or "stop"scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_MODEL, scope._response_model+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [finish_reason])scope._span.set_attribute(- SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, scope._finish_reason+ SemanticConvention.GEN_AI_OUTPUT_TYPE,+ "text" if isinstance(scope._llmresponse, str) else "json",)++ # Span Attributes for Tools+ # (Sarvam chat API may not expose tool_calls; omit or no-op if not present)+ if getattr(scope, "_tools", None):+ tools_list = scope._tools if isinstance(scope._tools, list) else [scope._tools]+ names = []+ ids = []+ args_list = []+ for t in tools_list:+ item = t if isinstance(t, dict) else {}+ fn = item.get("function") or {}+ if isinstance(fn, dict):+ names.append(fn.get("name", ""))+ args_list.append(str(fn.get("arguments", "")))+ else:+ names.append("")+ args_list.append("")+ ids.append(str(item.get("id", "")))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME, ",".join(names) if names else ""+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID, ",".join(ids) if ids else ""+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS,+ ",".join(args_list) if args_list else "",+ )++ # Span Attributes for Cost and Tokensscope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS,- scope._input_tokens + scope._output_tokens,+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,)+ scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- if cost:- scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)-- # Set span attributes for prompts and generations- if capture_message_content:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_INPUT_MESSAGES, formatted_messages+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # Record metrics- if not disable_metrics:- record_completion_metrics(- metrics=metrics,- gen_ai_operation=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,- GEN_AI_PROVIDER_NAME=SemanticConvention.GEN_AI_SYSTEM_SARVAM,+ # Span Attributes for Content+ input_msgs = build_input_messages(scope._kwargs.get("messages", []))+ output_msgs = build_output_messages(+ scope._llmresponse,+ getattr(scope, "_finish_reason", None),+ getattr(scope, "_tools", None),+ )+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ if capture_message_content and event_provider:+ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [finish_reason],+ "output_type": "text" if isinstance(scope._llmresponse, str) else "json",+ "temperature": scope._kwargs.get("temperature"),+ "max_tokens": scope._kwargs.get("max_tokens"),+ "top_p": scope._kwargs.get("top_p"),+ "frequency_penalty": scope._kwargs.get("frequency_penalty"),+ "presence_penalty": scope._kwargs.get("presence_penalty"),+ "stop_sequences": scope._kwargs.get("stop"),+ "seed": scope._kwargs.get("seed"),+ "choice_count": scope._kwargs.get("n", 1),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(scope, "_cache_read_input_tokens", 0),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ emit_inference_event(+ event_provider,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model,+ getattr(scope, "_response_model", request_model),+ input_messages=input_msgs,+ output_messages=output_msgs,server_address=scope._server_address,server_port=scope._server_port,- request_model=request_model,- response_model=scope._response_model,- environment=environment,- application_name=application_name,- start_time=scope._start_time,- end_time=scope._end_time,- cost=cost,- input_tokens=scope._input_tokens,- output_tokens=scope._output_tokens,- tbt=scope._tbt,- ttft=scope._ttft,+ **extra,)scope._span.set_status(Status(StatusCode.OK))+ # Span status and metrics+ if not disable_metrics:+ record_completion_metrics(+ metrics,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_SARVAM,+ scope._server_address,+ scope._server_port,+ request_model,+ getattr(scope, "_response_model", request_model),+ environment,+ application_name,+ scope._start_time,+ scope._end_time,+ input_tokens,+ output_tokens,+ cost,+ scope._tbt,+ scope._ttft,+ )+def process_streaming_chat_response(scope,@@ -263,19 +484,10 @@ def process_streaming_chat_response(capture_message_content,disable_metrics,version,+ event_provider=None,):"""Processes streaming chat response from Sarvam AI API.-- Args:- scope: The scope object containing span and response data.- pricing_info: Information about model pricing.- environment: The environment name.- application_name: The application name.- metrics: Metrics collection object.- capture_message_content: Whether to capture message content.- disable_metrics: Whether to disable metrics collection.- version: The version of the instrumentation."""common_chat_logic(scope,@@ -286,7 +498,8 @@ def process_streaming_chat_response(capture_message_content,disable_metrics,version,- True,+ is_stream=True,+ event_provider=event_provider,)@@ -304,31 +517,12 @@ def process_chat_response(capture_message_content,disable_metrics,version,+ event_provider=None,**kwargs,):"""Processes non-streaming chat response from Sarvam AI API.-- Args:- response: The API response object.- request_model: The model used for the request.- pricing_info: Information about model pricing.- server_port: The server port.- server_address: The server address.- environment: The environment name.- application_name: The application name.- metrics: Metrics collection object.- start_time: The request start time.- span: The OpenTelemetry span.- capture_message_content: Whether to capture message content.- disable_metrics: Whether to disable metrics collection.- version: The version of the instrumentation.- **kwargs: Additional keyword arguments.-- Returns:- The processed response object."""- # Create scope object for common processingscope = type("GenericScope", (), {})()scope._span = spanscope._start_time = start_time@@ -339,7 +533,6 @@ def process_chat_response(scope._tbt = 0scope._ttft = 0- # Initialize response attributesscope._llmresponse = ""scope._response_id = ""scope._response_model = request_model@@ -347,7 +540,6 @@ def process_chat_response(scope._input_tokens = 0scope._output_tokens = 0- # Extract response dataresponse_dict = response_as_dict(response)if response_dict.get("choices") and len(response_dict["choices"]) > 0:@@ -362,12 +554,19 @@ def process_chat_response(scope._input_tokens = usage.get("prompt_tokens", 0)scope._output_tokens = usage.get("completion_tokens", 0)+ # Handle token usage including reasoning tokens and cached tokens+ usage = response_dict.get("usage", {})+ prompt_tokens_details = usage.get("prompt_tokens_details", {})+ scope._cache_read_input_tokens = prompt_tokens_details.get("cached_tokens", 0)+ scope._cache_creation_input_tokens = (+ usage.get("cache_creation_input_tokens", 0) or 0+ )+if response_dict.get("id"):scope._response_id = response_dict["id"]if response_dict.get("model"):scope._response_model = response_dict["model"]- # Use common logic for processingcommon_chat_logic(scope,pricing_info,@@ -377,7 +576,8 @@ def process_chat_response(capture_message_content,disable_metrics,version,- False,+ is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/transformers/__init__.py b/sdk/python/src/openlit/instrumentation/transformers/__init__.pyindex 9297fda4..74d8df89 100644--- a/sdk/python/src/openlit/instrumentation/transformers/__init__.py+++ b/sdk/python/src/openlit/instrumentation/transformers/__init__.py@@ -28,6 +28,7 @@ class TransformersInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info")capture_message_content = kwargs.get("capture_message_content")disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("transformers")wrap_function_wrapper(@@ -42,6 +43,7 @@ class TransformersInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/transformers/transformers.py b/sdk/python/src/openlit/instrumentation/transformers/transformers.pyindex 6c600ce0..eb8eb3ba 100644--- a/sdk/python/src/openlit/instrumentation/transformers/transformers.py+++ b/sdk/python/src/openlit/instrumentation/transformers/transformers.py@@ -18,6 +18,7 @@ def pipeline_wrapper(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -56,6 +57,7 @@ def pipeline_wrapper(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/transformers/utils.py b/sdk/python/src/openlit/instrumentation/transformers/utils.pyindex 4acf7029..5f9762c1 100644--- a/sdk/python/src/openlit/instrumentation/transformers/utils.py+++ b/sdk/python/src/openlit/instrumentation/transformers/utils.py@@ -2,6 +2,8 @@HF Transformers OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -10,10 +12,13 @@ from openlit.__helpers import (general_tokens,get_chat_model_cost,common_span_attributes,+ otel_event,record_completion_metrics,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(content):"""@@ -65,6 +70,186 @@ def format_content(content):return str(content)+def build_input_messages(messages):+ """+ Convert Transformers request to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not messages:+ return []+ if isinstance(messages, str):+ return [{"role": "user", "parts": [{"type": "text", "content": messages}]}]+ if isinstance(messages, list) and len(messages) > 0:+ if (+ isinstance(messages[0], dict)+ and "role" in messages[0]+ and "content" in messages[0]+ ):+ otel_messages = []+ for message in messages:+ role = message.get("role", "user")+ content = message.get("content", "")+ if isinstance(content, list):+ parts = []+ for item in content:+ if isinstance(item, dict) and item.get("type") == "text":+ parts.append(+ {"type": "text", "content": item.get("text", "")}+ )+ if parts:+ otel_messages.append({"role": role, "parts": parts})+ elif content:+ otel_messages.append(+ {+ "role": role,+ "parts": [{"type": "text", "content": str(content)}],+ }+ )+ return (+ otel_messages+ if otel_messages+ else [+ {+ "role": "user",+ "parts": [{"type": "text", "content": str(messages)}],+ }+ ]+ )+ return [{"role": "user", "parts": [{"type": "text", "content": str(messages)}]}]+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Transformers response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ return [+ {+ "role": "assistant",+ "parts": parts,+ "finish_reason": finish_reason or "stop",+ }+ ]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if value is not None:+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = (+ value+ )+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = (+ value+ )+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = (+ value+ )+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "cache_read_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def common_chat_logic(scope,pricing_info,@@ -74,21 +259,21 @@ def common_chat_logic(capture_message_content,disable_metrics,version,- args,- kwargs,is_stream,+ event_provider=None,):"""- Process chat request and generate Telemetry+ Process chat request and generate Telemetry."""scope._end_time = time.time()- forward_params = scope._instance._forward_params+ forward_params = getattr(scope._instance, "_forward_params", {}) or {}request_model = scope._instance.model.config.name_or_path- input_tokens = general_tokens(scope._prompt)- output_tokens = general_tokens(scope._completion)+ input_tokens = getattr(scope, "_input_tokens", 0)+ output_tokens = getattr(scope, "_output_tokens", 0)+ # Compute costcost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributes@@ -108,10 +293,13 @@ def common_chat_logic(version,)- # Set request parameters from forward_params+ # Transformers pipeline (no additional API type attribute)++ # Span Attributes for Request parametersif forward_params.get("temperature") is not None:scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, forward_params["temperature"]+ SemanticConvention.GEN_AI_REQUEST_TEMPERATURE,+ forward_params["temperature"],)if forward_params.get("top_k") is not None:scope._span.set_attribute(@@ -123,10 +311,22 @@ def common_chat_logic()if forward_params.get("max_length") is not None:scope._span.set_attribute(- SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, forward_params["max_length"]+ SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,+ forward_params["max_length"],+ )++ # Span Attributes for Response parameters+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [getattr(scope, "_finish_reason", "stop")],+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, "text")- # Set token usage and cost attributes+ # Span Attributes for Cost and Tokensscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)@@ -134,36 +334,69 @@ def common_chat_logic(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,+ input_tokens + output_tokens,)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content- if capture_message_content:+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_INPUT_MESSAGES, scope._prompt+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._completion+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: scope._prompt,- },- )- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._completion,- },+ # Span Attributes for Content+ if capture_message_content:+ input_msgs = build_input_messages(scope._prompt)+ output_msgs = build_output_messages(+ scope._completion, getattr(scope, "_finish_reason", "stop"))+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)- scope._span.set_status(Status(StatusCode.OK))+ if event_provider:+ try:+ extra = {+ "response_id": getattr(scope, "_response_id", ""),+ "finish_reasons": [getattr(scope, "_finish_reason", "stop")],+ "output_type": "text",+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ if forward_params.get("temperature") is not None:+ extra["temperature"] = forward_params["temperature"]+ if forward_params.get("top_p") is not None:+ extra["top_p"] = forward_params["top_p"]+ if forward_params.get("max_length") is not None:+ extra["max_tokens"] = forward_params["max_length"]+ emit_inference_event(+ event_provider,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model,+ request_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)- # Record metrics using the standardized helper function+ # Span status and metrics+ scope._span.set_status(Status(StatusCode.OK))if not disable_metrics:record_completion_metrics(metrics,@@ -177,9 +410,9 @@ def common_chat_logic(application_name,scope._start_time,scope._end_time,- cost,input_tokens,output_tokens,+ cost,scope._tbt,scope._ttft,)@@ -202,6 +435,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,):"""Process chat request and generate Telemetry@@ -285,6 +519,14 @@ def process_chat_response(scope._tbt = 0scope._ttft = scope._end_time - scope._start_time+ # Handle token usage including reasoning tokens and cached tokens+ scope._input_tokens = general_tokens(scope._prompt)+ scope._output_tokens = general_tokens(scope._completion)+ scope._cache_read_input_tokens = 0+ scope._cache_creation_input_tokens = 0+ scope._response_id = ""+ scope._finish_reason = "stop"+common_chat_logic(scope,pricing_info,@@ -294,9 +536,8 @@ def process_chat_response(capture_message_content,disable_metrics,version,- args,- kwargs,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/vertexai/__init__.py b/sdk/python/src/openlit/instrumentation/vertexai/__init__.pyindex 9cb91a68..625f9369 100644--- a/sdk/python/src/openlit/instrumentation/vertexai/__init__.py+++ b/sdk/python/src/openlit/instrumentation/vertexai/__init__.py@@ -28,6 +28,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content = kwargs.get("capture_message_content", False)metrics = kwargs.get("metrics_dict")disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")# sync generative modelswrap_function_wrapper(@@ -42,6 +43,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -57,6 +59,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -73,6 +76,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -88,6 +92,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -104,6 +109,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -119,6 +125,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -135,6 +142,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)@@ -150,6 +158,7 @@ class VertexAIInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/vertexai/async_vertexai.py b/sdk/python/src/openlit/instrumentation/vertexai/async_vertexai.pyindex b02dbfb8..a90fb5bf 100644--- a/sdk/python/src/openlit/instrumentation/vertexai/async_vertexai.py+++ b/sdk/python/src/openlit/instrumentation/vertexai/async_vertexai.py@@ -29,6 +29,7 @@ def async_send_message(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for VertexAI AsyncMessages calls.@@ -56,6 +57,8 @@ def async_send_message(self._llmresponse = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._kwargs = kwargsself._start_time = time.time()self._end_time = None@@ -98,6 +101,7 @@ def async_send_message(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:handle_exception(self._span, e)@@ -148,6 +152,7 @@ def async_send_message(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/vertexai/utils.py b/sdk/python/src/openlit/instrumentation/vertexai/utils.pyindex 1ecacfcb..ba0deb44 100644--- a/sdk/python/src/openlit/instrumentation/vertexai/utils.py+++ b/sdk/python/src/openlit/instrumentation/vertexai/utils.py@@ -2,6 +2,8 @@VertexAI OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -9,12 +11,16 @@ from opentelemetry.trace import Status, StatusCodefrom openlit.__helpers import (calculate_ttft,calculate_tbt,+ general_tokens,get_chat_model_cost,record_completion_metrics,common_span_attributes,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def format_content(contents):"""@@ -58,6 +64,189 @@ def format_content(contents):return "\n".join(formatted_messages)+def build_input_messages(contents):+ """+ Convert Vertex AI request contents to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ if not contents:+ return []+ otel_messages = []+ for content in contents:+ try:+ role = (+ getattr(content, "role", "user")+ if not isinstance(content, dict)+ else content.get("role", "user")+ )+ parts_attr = (+ getattr(content, "parts", [])+ if not isinstance(content, dict)+ else content.get("parts", [])+ )+ parts_list = list(parts_attr) if parts_attr else []+ otel_parts = []+ for part in parts_list:+ text = (+ getattr(part, "text", None)+ if not isinstance(part, dict)+ else part.get("text")+ )+ if text:+ otel_parts.append({"type": "text", "content": str(text)})+ inline_data = (+ getattr(part, "inline_data", None)+ if not isinstance(part, dict)+ else part.get("inline_data")+ )+ if inline_data and not text:+ otel_parts.append({"type": "text", "content": "[inline_data]"})+ if otel_parts:+ otel_messages.append({"role": role, "parts": otel_parts})+ except Exception as e:+ logger.warning("Failed to process input content: %s", e, exc_info=True)+ continue+ return otel_messages+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert Vertex AI response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ try:+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ finish_reason_map = {+ "stop": "stop",+ "length": "length",+ "max_tokens": "length",+ "safety": "content_filter",+ "recitation": "content_filter",+ "other": "stop",+ }+ otel_finish = (+ finish_reason_map.get(str(finish_reason), str(finish_reason or "stop"))+ if finish_reason is not None+ else "stop"+ )+ return [{"role": "assistant", "parts": parts, "finish_reason": otel_finish}]+ except Exception as e:+ logger.warning("Failed to build output messages: %s", e, exc_info=True)+ return [{"role": "assistant", "parts": [], "finish_reason": "stop"}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {+ SemanticConvention.GEN_AI_OPERATION: operation_name,+ }+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port is not None:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id":+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons":+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type":+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "conversation_id":+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ elif key == "temperature":+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens":+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p":+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "frequency_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty":+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences":+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed":+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n"):+ if value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value+ elif key == "error_type":+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions":+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ else:+ attributes[key] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def process_chunk(scope, chunk):"""Process a chunk of response data and update state.@@ -73,8 +262,24 @@ def process_chunk(scope, chunk):# Aggregate response contentscope._llmresponse += str(chunk.text)- scope._input_tokens = chunk.usage_metadata.prompt_token_count- scope._output_tokens = chunk.usage_metadata.candidates_token_count+ # Handle token usage including reasoning tokens and cached tokens+ usage_metadata = getattr(chunk, "usage_metadata", None)+ scope._input_tokens = (+ getattr(usage_metadata, "prompt_token_count", 0) if usage_metadata else 0+ )+ scope._output_tokens = (+ getattr(usage_metadata, "candidates_token_count", 0) if usage_metadata else 0+ )+ scope._cache_read_input_tokens = (+ getattr(usage_metadata, "cached_content_token_count", 0)+ if usage_metadata+ else 0+ )+ scope._cache_creation_input_tokens = (+ getattr(usage_metadata, "cache_creation_input_tokens", 0)+ if usage_metadata+ else 0+ )def common_chat_logic(@@ -87,6 +292,7 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process chat request and generate Telemetry@@ -96,13 +302,26 @@ def common_chat_logic(if len(scope._timestamps) > 1:scope._tbt = calculate_tbt(scope._timestamps)- # Format content using VertexAI-specific logic- contents = scope._kwargs.get("contents", [])+ contents = scope._kwargs.get("contents", []) or (+ scope._args[0] if scope._args else []+ )formatted_messages = format_content(contents)- prompt = formatted_messages or str(scope._args[0][0])+ prompt = formatted_messages or (str(contents[0]) if contents else "")++ if (+ hasattr(scope, "_input_tokens")+ and scope._input_tokens is not None+ and hasattr(scope, "_output_tokens")+ and scope._output_tokens is not None+ ):+ input_tokens = scope._input_tokens+ output_tokens = scope._output_tokens+ else:+ input_tokens = general_tokens(prompt)+ output_tokens = general_tokens(scope._llmresponse)cost = get_chat_model_cost(- scope._request_model, pricing_info, scope._input_tokens, scope._output_tokens+ scope._request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributes@@ -113,7 +332,7 @@ def common_chat_logic(scope._server_address,scope._server_port,scope._request_model,- scope._request_model,+ getattr(scope, "_response_model", scope._request_model),environment,application_name,is_stream,@@ -122,10 +341,8 @@ def common_chat_logic(version,)- # Span Attributes for Request parameters (VertexAI-specific)- inference_config = scope._kwargs.get("generation_config", {})-- # List of attributes and their config keys+ # Span Attributes for Request parameters+ inference_config = scope._kwargs.get("generation_config", {}) or {}attributes = [(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, "frequency_penalty"),(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, "max_output_tokens"),@@ -135,52 +352,128 @@ def common_chat_logic((SemanticConvention.GEN_AI_REQUEST_TOP_P, "top_p"),(SemanticConvention.GEN_AI_REQUEST_TOP_K, "top_k"),]-- # Set each attribute if the corresponding value exists and is not Nonefor attribute, key in attributes:value = inference_config.get(key)if value is not None:scope._span.set_attribute(attribute, value)# Span Attributes for Response parameters+ if getattr(scope, "_response_id", None):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id+ )+ if getattr(scope, "_finish_reason", None) is not None:+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [scope._finish_reason],+ )scope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE,"text" if isinstance(scope._llmresponse, str) else "json",)+ # Span Attributes for Tools+ if getattr(scope, "_tools", None):+ tools = scope._tools+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_NAME,+ tools.get("name", "")+ if isinstance(tools, dict)+ else getattr(tools, "name", ""),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_CALL_ID,+ str(+ tools.get("id", "")+ if isinstance(tools, dict)+ else getattr(tools, "id", "")+ ),+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_TOOL_ARGS,+ str(+ tools.get("parameters", "")+ if isinstance(tools, dict)+ else getattr(tools, "parameters", "")+ ),+ )+# Span Attributes for Cost and Tokensscope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens+ SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, scope._output_tokens+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)scope._span.set_attribute(- SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE,- scope._input_tokens + scope._output_tokens,+ SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens)scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,+ )+ if hasattr(scope, "_cache_creation_input_tokens"):scope._span.set_attribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES, scope._llmresponse+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,)- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ # Span Attributes for Content+ if capture_message_content:+ input_msgs = build_input_messages(+ scope._kwargs.get("contents", []) or (scope._args[0] if scope._args else []))- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: scope._llmresponse,- },+ output_msgs = build_output_messages(+ scope._llmresponse,+ getattr(scope, "_finish_reason", None),+ tool_calls=getattr(scope, "_tools", None),)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)++ if event_provider:+ try:+ gen_config = scope._kwargs.get("generation_config", {}) or {}+ extra = {+ "cache_read_input_tokens": getattr(+ scope, "_cache_read_input_tokens", 0+ ),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [getattr(scope, "_finish_reason", "")],+ "output_type": "text"+ if isinstance(scope._llmresponse, str)+ else "json",+ "temperature": gen_config.get("temperature"),+ "max_tokens": gen_config.get("max_output_tokens"),+ "top_p": gen_config.get("top_p"),+ "frequency_penalty": gen_config.get("frequency_penalty"),+ "presence_penalty": gen_config.get("presence_penalty"),+ "stop_sequences": gen_config.get("stop_sequences"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ }+ emit_inference_event(+ event_provider=event_provider,+ operation_name=SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model=scope._request_model,+ response_model=getattr(+ scope, "_response_model", scope._request_model+ ),+ input_messages=input_msgs,+ output_messages=output_msgs,+ tool_definitions=None,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,+ )+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)scope._span.set_status(Status(StatusCode.OK))@@ -193,13 +486,13 @@ def common_chat_logic(scope._server_address,scope._server_port,scope._request_model,- scope._request_model,+ getattr(scope, "_response_model", scope._request_model),environment,application_name,scope._start_time,scope._end_time,- scope._input_tokens,- scope._output_tokens,+ input_tokens,+ output_tokens,cost,scope._tbt,scope._ttft,@@ -215,6 +508,7 @@ def process_streaming_chat_response(capture_message_content=False,disable_metrics=False,version="",+ event_provider=None,):"""Process streaming chat response and generate telemetry.@@ -230,6 +524,7 @@ def process_streaming_chat_response(disable_metrics,version,is_stream=True,+ event_provider=event_provider,)@@ -247,6 +542,7 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,**kwargs,):"""@@ -258,15 +554,42 @@ def process_chat_response(scope._start_time = start_timescope._end_time = time.time()scope._span = span- scope._llmresponse = response.text- scope._input_tokens = response.usage_metadata.prompt_token_count- scope._output_tokens = response.usage_metadata.candidates_token_count+ scope._llmresponse = getattr(response, "text", "") or ""+ # Handle token usage including reasoning tokens and cached tokens+ usage_metadata = getattr(response, "usage_metadata", None)+ scope._input_tokens = (+ getattr(usage_metadata, "prompt_token_count", 0) if usage_metadata else 0+ )+ scope._output_tokens = (+ getattr(usage_metadata, "candidates_token_count", 0) if usage_metadata else 0+ )+ scope._cache_read_input_tokens = (+ getattr(usage_metadata, "cached_content_token_count", 0)+ if usage_metadata+ else 0+ )+ scope._cache_creation_input_tokens = (+ getattr(usage_metadata, "cache_creation_input_tokens", 0)+ if usage_metadata+ else 0+ )scope._timestamps = []scope._ttft, scope._tbt = scope._end_time - scope._start_time, 0scope._server_address, scope._server_port = server_address, server_portscope._request_model = request_model+ scope._response_model = request_modelscope._kwargs = kwargsscope._args = [kwargs.get("contents", [])]+ candidates = getattr(response, "candidates", None) or []+ first_candidate = candidates[0] if candidates else None+ scope._finish_reason = (+ str(getattr(first_candidate, "finish_reason", None) or "")+ if first_candidate is not None+ else ""+ )+ scope._response_id = getattr(response, "id", None) or getattr(+ response, "name", None+ )common_chat_logic(scope,@@ -278,6 +601,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/vertexai/vertexai.py b/sdk/python/src/openlit/instrumentation/vertexai/vertexai.pyindex bc4028bf..c77d37f3 100644--- a/sdk/python/src/openlit/instrumentation/vertexai/vertexai.py+++ b/sdk/python/src/openlit/instrumentation/vertexai/vertexai.py@@ -29,6 +29,7 @@ def send_message(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for VertexAI messages to collect metrics.@@ -56,6 +57,8 @@ def send_message(self._llmresponse = ""self._input_tokens = 0self._output_tokens = 0+ self._cache_read_input_tokens = 0+ self._cache_creation_input_tokens = 0self._kwargs = kwargsself._start_time = time.time()self._end_time = None@@ -98,6 +101,7 @@ def send_message(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:handle_exception(self._span, e)@@ -148,6 +152,7 @@ def send_message(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,**kwargs,)diff --git a/sdk/python/src/openlit/instrumentation/vllm/__init__.py b/sdk/python/src/openlit/instrumentation/vllm/__init__.pyindex eeab327a..0a2cd6d1 100644--- a/sdk/python/src/openlit/instrumentation/vllm/__init__.py+++ b/sdk/python/src/openlit/instrumentation/vllm/__init__.py@@ -26,6 +26,7 @@ class VLLMInstrumentor(BaseInstrumentor):pricing_info = kwargs.get("pricing_info", {})capture_message_content = kwargs.get("capture_message_content", False)disable_metrics = kwargs.get("disable_metrics")+ event_provider = kwargs.get("event_provider")version = importlib.metadata.version("vllm")# Chat completions@@ -41,6 +42,7 @@ class VLLMInstrumentor(BaseInstrumentor):capture_message_content,metrics,disable_metrics,+ event_provider=event_provider,),)diff --git a/sdk/python/src/openlit/instrumentation/vllm/utils.py b/sdk/python/src/openlit/instrumentation/vllm/utils.pyindex b58a01f1..4cfef1a5 100644--- a/sdk/python/src/openlit/instrumentation/vllm/utils.py+++ b/sdk/python/src/openlit/instrumentation/vllm/utils.py@@ -2,6 +2,8 @@vLLM OpenTelemetry instrumentation utility functions"""+import json+import loggingimport timefrom opentelemetry.trace import Status, StatusCode@@ -11,9 +13,12 @@ from openlit.__helpers import (get_chat_model_cost,common_span_attributes,record_completion_metrics,+ otel_event,)from openlit.semcov import SemanticConvention+logger = logging.getLogger(__name__)+def get_inference_config(args, kwargs):"""@@ -40,6 +45,147 @@ def format_content(prompts):return str(prompts)+def build_input_messages(prompts):+ """+ Convert vLLM request prompts to OTel input message structure.+ Follows gen-ai-input-messages schema.+ """+ content = format_content(prompts) if prompts else ""+ return [{"role": "user", "parts": [{"type": "text", "content": content}]}]+++def build_output_messages(response_text, finish_reason, tool_calls=None):+ """+ Convert vLLM response to OTel output message structure.+ Follows gen-ai-output-messages schema.+ """+ parts = []+ if response_text:+ parts.append({"type": "text", "content": str(response_text)})+ if tool_calls:+ tools = tool_calls if isinstance(tool_calls, list) else [tool_calls]+ for t in tools:+ tc = t if isinstance(t, dict) else {}+ fn = tc.get("function", {}) or {}+ parts.append(+ {+ "type": "tool_call",+ "id": tc.get("id", ""),+ "name": fn.get("name", ""),+ "arguments": fn.get("arguments", ""),+ }+ )+ reason = finish_reason if finish_reason else "stop"+ return [{"role": "assistant", "parts": parts, "finish_reason": reason}]+++def _set_span_messages_as_array(span, input_messages, output_messages):+ """Set gen_ai.input.messages and gen_ai.output.messages on span as JSON array strings (OTel)."""+ try:+ if input_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_INPUT_MESSAGES,+ json.dumps(input_messages)+ if isinstance(input_messages, list)+ else input_messages,+ )+ if output_messages is not None:+ span.set_attribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ json.dumps(output_messages)+ if isinstance(output_messages, list)+ else output_messages,+ )+ except Exception as e:+ logger.warning("Failed to set span message attributes: %s", e, exc_info=True)+++def emit_inference_event(+ event_provider,+ operation_name,+ request_model,+ response_model,+ input_messages=None,+ output_messages=None,+ tool_definitions=None,+ server_address=None,+ server_port=None,+ **extra_attrs,+):+ """+ Emit gen_ai.client.inference.operation.details event.+ """+ try:+ if not event_provider:+ return+ attributes = {SemanticConvention.GEN_AI_OPERATION: operation_name}+ if request_model:+ attributes[SemanticConvention.GEN_AI_REQUEST_MODEL] = request_model+ if response_model:+ attributes[SemanticConvention.GEN_AI_RESPONSE_MODEL] = response_model+ if server_address:+ attributes[SemanticConvention.SERVER_ADDRESS] = server_address+ if server_port:+ attributes[SemanticConvention.SERVER_PORT] = server_port+ if input_messages is not None:+ attributes[SemanticConvention.GEN_AI_INPUT_MESSAGES] = input_messages+ if output_messages is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_MESSAGES] = output_messages+ if tool_definitions is not None:+ attributes[SemanticConvention.GEN_AI_TOOL_DEFINITIONS] = tool_definitions+ for key, value in extra_attrs.items():+ if key == "response_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_ID] = value+ elif key == "finish_reasons" and value is not None:+ attributes[SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON] = value+ elif key == "output_type" and value is not None:+ attributes[SemanticConvention.GEN_AI_OUTPUT_TYPE] = value+ elif key == "temperature" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TEMPERATURE] = value+ elif key == "max_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS] = value+ elif key == "top_p" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_P] = value+ elif key == "top_k" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_TOP_K] = value+ elif key == "frequency_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY] = value+ elif key == "presence_penalty" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY] = value+ elif key == "stop_sequences" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES] = value+ elif key == "seed" and value is not None:+ attributes[SemanticConvention.GEN_AI_REQUEST_SEED] = value+ elif key in ("choice_count", "n") and value is not None and value != 1:+ attributes[SemanticConvention.GEN_AI_REQUEST_CHOICE_COUNT] = value+ elif key == "input_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS] = value+ elif key == "output_tokens" and value is not None:+ attributes[SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS] = value+ elif key == "cache_read_input_tokens":+ attributes[SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS] = (+ value if value is not None else 0+ )+ elif key == "cache_creation_input_tokens":+ attributes[+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS+ ] = value if value is not None else 0+ elif key == "error_type" and value is not None:+ attributes[SemanticConvention.ERROR_TYPE] = value+ elif key == "system_instructions" and value is not None:+ attributes[SemanticConvention.GEN_AI_SYSTEM_INSTRUCTIONS] = value+ elif key == "conversation_id" and value is not None:+ attributes[SemanticConvention.GEN_AI_CONVERSATION_ID] = value+ event = otel_event(+ name=SemanticConvention.GEN_AI_CLIENT_INFERENCE_OPERATION_DETAILS,+ attributes=attributes,+ body="",+ )+ event_provider.emit(event)+ except Exception as e:+ logger.warning("Failed to emit inference event: %s", e, exc_info=True)++def common_chat_logic(scope,pricing_info,@@ -50,26 +196,36 @@ def common_chat_logic(disable_metrics,version,is_stream,+ event_provider=None,):"""Process chat request and generate Telemetry"""request_model = scope._request_model+ response_model = getattr(scope, "_response_model", request_model)- # Extract prompts and completions from vLLM response+ # Extract prompts and completions from vLLM response; set scope._input_tokens / _output_tokensinput_tokens = 0output_tokens = 0prompt = ""completion = ""+ finish_reason = "stop"for output in scope._response:prompt += output.prompt + "\n"if output.outputs and len(output.outputs) > 0:- completion += output.outputs[0].text + "\n"- input_tokens += general_tokens(output.prompt)- output_tokens += general_tokens(output.outputs[0].text)+ out = output.outputs[0]+ completion += out.text + "\n"+ if finish_reason == "stop" and getattr(out, "finish_reason", None):+ finish_reason = out.finish_reason+ input_tokens += general_tokens(output.prompt)+ output_tokens += general_tokens(out.text)+ scope._input_tokens = input_tokens+ scope._output_tokens = output_tokens++ # Compute costcost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributes@@ -80,7 +236,7 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- request_model,+ response_model,environment,application_name,is_stream,@@ -122,6 +278,9 @@ def common_chat_logic()# Span Attributes for Response parameters+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [finish_reason]+ )scope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, "text")# Span Attributes for Cost and Tokens@@ -136,28 +295,62 @@ def common_chat_logic()scope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)- # Span Attributes for Content- if capture_message_content:- scope._span.set_attribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt)- scope._span.set_attribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, completion)-- # To be removed once the change to span_attributes (from span events) is complete- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_PROMPT_EVENT,- attributes={- SemanticConvention.GEN_AI_INPUT_MESSAGES: prompt,- },+ # OTel cached token attributes (set even when 0)+ if hasattr(scope, "_cache_read_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS,+ scope._cache_read_input_tokens,)- scope._span.add_event(- name=SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT,- attributes={- SemanticConvention.GEN_AI_OUTPUT_MESSAGES: completion,- },+ if hasattr(scope, "_cache_creation_input_tokens"):+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS,+ scope._cache_creation_input_tokens,+ )++ # Span Attributes for Content+ input_msgs = build_input_messages(+ scope._kwargs.get("prompts") or (scope._args[0] if scope._args else "")+ )+ output_msgs = build_output_messages(completion, finish_reason)+ _set_span_messages_as_array(scope._span, input_msgs, output_msgs)+ if capture_message_content and event_provider:++ def _get_attr(obj, name, default=None):+ return getattr(obj, name, default) if obj else default++ extra = {+ "response_id": getattr(scope, "_response_id", None),+ "finish_reasons": [finish_reason],+ "output_type": "text",+ "temperature": _get_attr(inference_config, "temperature"),+ "max_tokens": _get_attr(inference_config, "max_tokens"),+ "top_p": _get_attr(inference_config, "top_p"),+ "top_k": _get_attr(inference_config, "top_k"),+ "frequency_penalty": _get_attr(inference_config, "frequency_penalty"),+ "presence_penalty": _get_attr(inference_config, "presence_penalty"),+ "stop_sequences": _get_attr(inference_config, "stop_sequences"),+ "input_tokens": input_tokens,+ "output_tokens": output_tokens,+ "cache_read_input_tokens": getattr(scope, "_cache_read_input_tokens", 0),+ "cache_creation_input_tokens": getattr(+ scope, "_cache_creation_input_tokens", 0+ ),+ }+ emit_inference_event(+ event_provider,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ request_model,+ response_model,+ input_messages=input_msgs,+ output_messages=output_msgs,+ server_address=scope._server_address,+ server_port=scope._server_port,+ **extra,)scope._span.set_status(Status(StatusCode.OK))- # Metrics+ # Span status and metricsif not disable_metrics:record_completion_metrics(metrics,@@ -166,7 +359,7 @@ def common_chat_logic(scope._server_address,scope._server_port,request_model,- request_model,+ response_model,environment,application_name,scope._start_time,@@ -196,12 +389,12 @@ def process_chat_response(capture_message_content=False,disable_metrics=False,version="1.0.0",+ event_provider=None,):"""Process chat request and generate Telemetry"""- # Create scope objectscope = type("GenericScope", (), {})()scope._response = response@@ -216,6 +409,10 @@ def process_chat_response(scope._args = argsscope._kwargs = kwargs+ # Handle token usage including reasoning tokens and cached tokens+ scope._cache_read_input_tokens = 0+ scope._cache_creation_input_tokens = 0+common_chat_logic(scope,pricing_info,@@ -226,6 +423,7 @@ def process_chat_response(disable_metrics,version,is_stream=False,+ event_provider=event_provider,)return responsediff --git a/sdk/python/src/openlit/instrumentation/vllm/vllm.py b/sdk/python/src/openlit/instrumentation/vllm/vllm.pyindex 9d75884c..ddba1ec0 100644--- a/sdk/python/src/openlit/instrumentation/vllm/vllm.py+++ b/sdk/python/src/openlit/instrumentation/vllm/vllm.py@@ -18,6 +18,7 @@ def generate(capture_message_content,metrics,disable_metrics,+ event_provider=None,):"""Generates a telemetry wrapper for GenAI function call@@ -56,6 +57,7 @@ def generate(capture_message_content=capture_message_content,disable_metrics=disable_metrics,version=version,+ event_provider=event_provider,)except Exception as e:diff --git a/sdk/python/src/openlit/semcov/__init__.py b/sdk/python/src/openlit/semcov/__init__.pyindex 9788012d..bb6cc132 100644--- a/sdk/python/src/openlit/semcov/__init__.py+++ b/sdk/python/src/openlit/semcov/__init__.py@@ -8,6 +8,51 @@ The purpose is to standardize the semantics for easier integration, analytics, a"""+# Helper function to safely extract OTel attribute values+def _get_otel_attr(obj, attr_name, fallback_string):+ """+ Safely extract attribute value from OTel semantic convention object.+ Handles both direct string attributes and objects with .value property.+ """+ if obj is None:+ return fallback_string++ try:+ attr = getattr(obj, attr_name, None)+ if attr is None:+ return fallback_string+ # If it has .value property (some versions), use that+ if hasattr(attr, "value"):+ return attr.value+ # Otherwise use the attribute directly (it's already a string)+ return str(attr)+ except Exception:+ return fallback_string+++# Import official OpenTelemetry semantic conventions (TIER 1)+HAS_OFFICIAL_OTEL_SEMCONV = False+OTelGenAIAttributes = None+OTelServerAttributes = None+OTelErrorAttributes = None++try:+ from opentelemetry.semconv._incubating.attributes import (+ gen_ai_attributes as OTelGenAIAttributes,+ )+ from opentelemetry.semconv._incubating.attributes import (+ server_attributes as OTelServerAttributes,+ )+ from opentelemetry.semconv._incubating.attributes import (+ error_attributes as OTelErrorAttributes,+ )++ HAS_OFFICIAL_OTEL_SEMCONV = True+except ImportError:+ # Fallback if opentelemetry-semconv is not installed+ pass++class SemanticConvention:"""The SemanticConvention class provides a centralized repository of constant values that@@ -16,12 +61,24 @@ class SemanticConvention:parameters, usage metrics, response attributes, and integrations with external AI anddatabase systems. It is designed to facilitate consistency and understandability acrossthe application's data logging and processing functionalities.++ TIER 1: Official OTel Standard Attributes (imported from opentelemetry.semconv)+ TIER 2: OpenAI-Specific and Provider-Specific Attributes+ TIER 3: OpenLIT Custom Extensions (cost tracking, metrics, etc.)"""- # General Attributes (OTel Semconv)- SERVER_PORT = "server.port"- SERVER_ADDRESS = "server.address"- ERROR_TYPE = "error.type"+ # ==========================================+ # TIER 1: Official OTel Standard Attributes+ # ==========================================++ # Server Attributes (OTel Semconv)+ SERVER_ADDRESS = _get_otel_attr(+ OTelServerAttributes, "SERVER_ADDRESS", "server.address"+ )+ SERVER_PORT = _get_otel_attr(OTelServerAttributes, "SERVER_PORT", "server.port")++ # Error Attributes (OTel Semconv)+ ERROR_TYPE = _get_otel_attr(OTelErrorAttributes, "ERROR_TYPE", "error.type")# GenAI Metric Names (OTel Semconv)GEN_AI_CLIENT_TOKEN_USAGE = "gen_ai.client.token.usage"@@ -50,43 +107,91 @@ class SemanticConvention:GEN_AI_TOOL_DEFINITIONS = "gen_ai.tool.definitions"# GenAI Request Attributes (OTel Semconv)- GEN_AI_OPERATION = "gen_ai.operation.name"- GEN_AI_PROVIDER_NAME = "gen_ai.provider.name"- GEN_AI_OUTPUT_TYPE = "gen_ai.output.type"- GEN_AI_ENDPOINT = "gen_ai.endpoint"- GEN_AI_REQUEST_MODEL = "gen_ai.request.model"- GEN_AI_REQUEST_SEED = "gen_ai.request.seed"- GEN_AI_REQUEST_CHOICE_COUNT = "gen_ai.request.choice.count"- GEN_AI_REQUEST_ENCODING_FORMATS = "gen_ai.request.encoding_formats"- GEN_AI_REQUEST_FREQUENCY_PENALTY = "gen_ai.request.frequency_penalty"- GEN_AI_REQUEST_MAX_TOKENS = "gen_ai.request.max_tokens"- GEN_AI_REQUEST_PRESENCE_PENALTY = "gen_ai.request.presence_penalty"- GEN_AI_REQUEST_STOP_SEQUENCES = "gen_ai.request.stop_sequences"- GEN_AI_REQUEST_TEMPERATURE = "gen_ai.request.temperature"- GEN_AI_REQUEST_TOP_K = "gen_ai.request.top_k"- GEN_AI_REQUEST_TOP_P = "gen_ai.request.top_p"-- # GenAI Conversation Attributes (OTel Semconv)- GEN_AI_CONVERSATION_ID = "gen_ai.conversation.id"-- # GenAI Response Attributes (OTel Semconv)- GEN_AI_TOKEN_TYPE = "gen_ai.token.type"- GEN_AI_RESPONSE_FINISH_REASON = "gen_ai.response.finish_reasons"- GEN_AI_RESPONSE_ID = "gen_ai.response.id"- GEN_AI_RESPONSE_MODEL = "gen_ai.response.model"-- GEN_AI_USAGE_INPUT_TOKENS = "gen_ai.usage.input_tokens"- GEN_AI_USAGE_OUTPUT_TOKENS = "gen_ai.usage.output_tokens"- GEN_AI_USAGE_REASONING_TOKENS = "gen_ai.usage.reasoning_tokens"- GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS = (- "gen_ai.usage.cache_creation.input_tokens"+ GEN_AI_OPERATION = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_OPERATION_NAME", "gen_ai.operation.name"+ )+ GEN_AI_PROVIDER_NAME = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_PROVIDER_NAME", "gen_ai.provider.name"+ )+ GEN_AI_OUTPUT_TYPE = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_OUTPUT_TYPE", "gen_ai.output.type"+ )+ GEN_AI_REQUEST_MODEL = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_REQUEST_MODEL", "gen_ai.request.model"+ )+ GEN_AI_REQUEST_SEED = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_REQUEST_SEED", "gen_ai.request.seed"+ )+ GEN_AI_REQUEST_CHOICE_COUNT = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_REQUEST_CHOICE_COUNT",+ "gen_ai.request.choice.count",+ )+ GEN_AI_REQUEST_ENCODING_FORMATS = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_REQUEST_ENCODING_FORMATS",+ "gen_ai.request.encoding_formats",+ )+ GEN_AI_EMBEDDINGS_DIMENSION_COUNT = "gen_ai.embeddings.dimension.count"+ GEN_AI_REQUEST_FREQUENCY_PENALTY = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_REQUEST_FREQUENCY_PENALTY",+ "gen_ai.request.frequency_penalty",+ )+ GEN_AI_REQUEST_MAX_TOKENS = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_REQUEST_MAX_TOKENS", "gen_ai.request.max_tokens"+ )+ GEN_AI_REQUEST_PRESENCE_PENALTY = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_REQUEST_PRESENCE_PENALTY",+ "gen_ai.request.presence_penalty",+ )+ GEN_AI_REQUEST_STOP_SEQUENCES = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_REQUEST_STOP_SEQUENCES",+ "gen_ai.request.stop_sequences",+ )+ GEN_AI_REQUEST_TEMPERATURE = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_REQUEST_TEMPERATURE", "gen_ai.request.temperature"+ )+ GEN_AI_REQUEST_TOP_K = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_REQUEST_TOP_K", "gen_ai.request.top_k"+ )+ GEN_AI_REQUEST_TOP_P = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_REQUEST_TOP_P", "gen_ai.request.top_p"+ )+ GEN_AI_CONVERSATION_ID = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_CONVERSATION_ID", "gen_ai.conversation.id"+ )+ GEN_AI_RESPONSE_FINISH_REASON = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_RESPONSE_FINISH_REASONS",+ "gen_ai.response.finish_reasons",+ )+ GEN_AI_RESPONSE_ID = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_RESPONSE_ID", "gen_ai.response.id"+ )+ GEN_AI_RESPONSE_MODEL = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_RESPONSE_MODEL", "gen_ai.response.model"+ )+ GEN_AI_USAGE_INPUT_TOKENS = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_USAGE_INPUT_TOKENS", "gen_ai.usage.input_tokens"+ )+ GEN_AI_USAGE_OUTPUT_TOKENS = _get_otel_attr(+ OTelGenAIAttributes, "GEN_AI_USAGE_OUTPUT_TOKENS", "gen_ai.usage.output_tokens"+ )+ GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS",+ "gen_ai.usage.cache_creation.input_tokens",+ )+ GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS = _get_otel_attr(+ OTelGenAIAttributes,+ "GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS",+ "gen_ai.usage.cache_read.input_tokens",)- GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS = "gen_ai.usage.cache_read.input_tokens"- GEN_AI_USAGE_READ_UNITS = "gen_ai.usage.read_units"- GEN_AI_USAGE_RERANK_UNITS = "gen_ai.usage.rerank_units"- GEN_AI_TOOL_CALL_ID = "gen_ai.tool.call.id"- GEN_AI_TOOL_NAME = "gen_ai.tool.name"- GEN_AI_TOOL_ARGS = "gen_ai.tool.args"++ GEN_AI_ENDPOINT = "gen_ai.endpoint"# GenAI Operation Types (OTel Semconv)GEN_AI_OPERATION_TYPE_TEXT_COMPLETION = "text_completion"@@ -149,10 +254,21 @@ class SemanticConvention:GEN_AI_SYSTEM_VERTEXAI = "vertex_ai"GEN_AI_SYSTEM_XAI = "xai"- # GenAI OpenAI Attributes (OTel Semconv)- GEN_AI_REQUEST_SERVICE_TIER = "gen_ai.request.service_tier"- GEN_AI_RESPONSE_SERVICE_TIER = "gen_ai.response.service_tier"- GEN_AI_RESPONSE_SYSTEM_FINGERPRINT = "gen_ai.response.system_fingerprint"+ # ==========================================+ # TIER 2: OpenAI-Specific OTel Attributes+ # ==========================================++ # From official OTel OpenAI convention documentation+ # https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/openai.md+ OPENAI_REQUEST_SERVICE_TIER = "openai.request.service_tier"+ OPENAI_RESPONSE_SERVICE_TIER = "openai.response.service_tier"+ OPENAI_RESPONSE_SYSTEM_FINGERPRINT = "openai.response.system_fingerprint"+ OPENAI_API_TYPE = "openai.api.type" # "chat_completions" or "responses"++ # Backward compatibility aliases (deprecated, use OPENAI_* instead)+ GEN_AI_REQUEST_SERVICE_TIER = OPENAI_REQUEST_SERVICE_TIER+ GEN_AI_RESPONSE_SERVICE_TIER = OPENAI_RESPONSE_SERVICE_TIER+ GEN_AI_RESPONSE_SYSTEM_FINGERPRINT = OPENAI_RESPONSE_SYSTEM_FINGERPRINT# GenAI System Names (Extra)GEN_AI_SYSTEM_HUGGING_FACE = "huggingface"@@ -329,9 +445,17 @@ class SemanticConvention:GEN_AI_APPLICATION_NAME = "gen_ai.application_name"GEN_AI_SDK_VERSION = "gen_ai.sdk.version"- # GenAI Response Attributes (Extra)+ # ==========================================+ # TIER 3: OpenLIT Custom Extensions+ # ==========================================+ # These attributes are specific to OpenLIT and not part of the OTel semantic+ # conventions specification. They extend the standard conventions for cost+ # tracking, metrics, framework integration, and other openlit-specific features.++ # GenAI Response Attributes (Extra/Custom)GEN_AI_USAGE_TOTAL_TOKENS = "gen_ai.usage.total_tokens"GEN_AI_USAGE_COST = "gen_ai.usage.cost"+ GEN_AI_USAGE_REASONING_TOKENS = "gen_ai.usage.reasoning_tokens"GEN_AI_RESPONSE_IMAGE = "gen_ai.response.image"GEN_AI_TOOL_CALLS = "gen_ai.response.tool_calls"@@ -346,6 +470,9 @@ class SemanticConvention:# Tool Attributes (LangChain/Framework Support)GEN_AI_TOOL_INPUT = "gen_ai.tool.input"GEN_AI_TOOL_OUTPUT = "gen_ai.tool.output"+ GEN_AI_TOOL_NAME = "gen_ai.tool.name"+ GEN_AI_TOOL_CALL_ID = "gen_ai.tool.call.id"+ GEN_AI_TOOL_ARGS = "gen_ai.tool.args"# Retrieval Attributes (LangChain Retriever Support)GEN_AI_RETRIEVAL_QUERY = "gen_ai.retrieval.query"@@ -955,8 +1082,8 @@ class SemanticConvention:# === ENHANCED SEMANTIC CONVENTIONS FOR COMPREHENSIVE INSTRUMENTATION ===# Message structure attributes (reuse existing prompt for input, add output messages)- # Note: For input messages, we reuse GEN_AI_INPUT_MESSAGES for consistency- GEN_AI_OUTPUT_MESSAGES = "gen_ai.output_messages"+ # Note: OTel spec uses gen_ai.output.messages (with dot); do not override with wrong key+ # GEN_AI_OUTPUT_MESSAGES is defined above as "gen_ai.output.messages"GEN_AI_MESSAGE_ROLE = "gen_ai.message.role"GEN_AI_MESSAGE_CONTENT = "gen_ai.message.content"