Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
[Feat]: Refactor OpenAI Agents OpenTelemetry Instrumentation (#807)
Patcherab7330f49bc8
diff --git a/contributors/framework-instrumentation-guide.md b/contributors/framework-instrumentation-guide.mdindex 03e49646..7d989f0b 100644--- a/contributors/framework-instrumentation-guide.md+++ b/contributors/framework-instrumentation-guide.md@@ -1,13 +1,13 @@# Framework Instrumentation Guide for OpenLIT-This guide provides a comprehensive, step-by-step process for adding new framework instrumentations or updating existing ones in OpenLIT. It's based on the optimized Haystack instrumentation and ensures consistency, performance, and competitive advantages across all framework integrations.+This guide provides a comprehensive, step-by-step process for adding new framework instrumentations or updating existing ones in OpenLIT. It's based on lessons learned from successful instrumentations including OpenAI Agents, Haystack, and others, ensuring consistency, performance, and competitive advantages across all framework integrations.## Table of Contents1. [Prerequisites](#prerequisites)2. [Phase 1: Competitive Analysis](#phase-1-competitive-analysis)3. [Phase 2: Framework Analysis](#phase-2-framework-analysis)-4. [Phase 3: Implementation Structure](#phase-3-implementation-structure)+4. [Phase 3: Implementation Strategy](#phase-3-implementation-strategy)5. [Phase 4: Code Implementation](#phase-4-code-implementation)6. [Phase 5: Testing & Validation](#phase-5-testing--validation)7. [Phase 6: Optimization](#phase-6-optimization)@@ -23,17 +23,23 @@ Before starting, ensure you have:- Access to competitor repositories for analysis- Understanding of OpenTelemetry concepts+**CRITICAL**: Always use local OpenLIT source code for testing:+```bash+# DON'T install OpenLIT - use local source+PYTHONPATH=sdk/python/src python your_test_script.py+```+## Phase 1: Competitive Analysis-### Step 1.1: Research Competitor Implementations+### Step 1.1: Clone and Research Competitor Implementations**Primary Competitors:**- [OpenInference](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation)- [OpenLLMetry](https://github.com/traceloop/openllmetry/tree/main/packages)-**Research Tasks:**+**Research Process:**```bash-# Clone competitor repositories for analysis+# Clone competitor repositories for deep analysisgit clone https://github.com/Arize-ai/openinference.gitgit clone https://github.com/traceloop/openllmetry.git@@ -42,661 +48,710 @@ cd openinference/python/instrumentation/openinference-instrumentation-{frameworkcd openllmetry/packages/openllmetry-instrumentation-{framework}```-**Analysis Checklist:**-- [ ] **Span Structure**: How many spans do they create?-- [ ] **Span Hierarchy**: Do they maintain parent-child relationships?-- [ ] **Attributes**: What attributes do they capture?-- [ ] **Content Capture**: Do they capture input/output content?-- [ ] **Metrics**: What metrics do they track?+**Deep Analysis Checklist:**+- [ ] **Integration Pattern**: Function wrapping vs native integration (like TracingProcessor)+- [ ] **Span Structure**: How many spans do they create and why?+- [ ] **Span Hierarchy**: Do they maintain proper parent-child relationships?+- [ ] **Span Naming**: What naming convention do they use?+- [ ] **Attributes**: What attributes do they capture? Check against semantic conventions+- [ ] **Content Capture**: Do they capture input/output content with MIME types?+- [ ] **Business Intelligence**: Do they track cost, tokens, performance metrics?+- [ ] **Error Handling**: How do they handle framework version differences?- [ ] **Performance**: How much overhead do they add?- [ ] **Coverage**: Which framework operations do they instrument?-### Step 1.2: Document Competitor Strengths & Gaps+### Step 1.2: Document Competitive Gaps and OpenLIT Advantages-Create a comparison table:+Create detailed comparison:| Feature | OpenInference | OpenLLMetry | OpenLIT Target ||---------|---------------|-------------|----------------|-| Span Count | X spans | Y spans | Z spans (optimal) |-| Technical Detail | High/Medium/Low | High/Medium/Low | Enhanced |-| Business Intelligence | High/Medium/Low | High/Medium/Low | Complete |-| Content Capture | Yes/No/Partial | Yes/No/Partial | Full |-| Cost Tracking | Yes/No | Yes/No | Yes |-| Performance Impact | X ms | Y ms | Minimal |+| Integration Method | Function wrapping | Function wrapping | Native (if available) |+| Span Count | X spans | Y spans | Z+ spans (comprehensive) |+| Business Intelligence | None/Basic | None/Basic | **Complete** (cost, tokens, metrics) |+| Content Capture | Basic | Basic | **Enhanced** (MIME types, structured) |+| Cross-System Tracing | Limited | Limited | **Full** (links to LLM providers) |+| Error Resilience | Basic | Basic | **Robust** (graceful degradation) |+| Semantic Conventions | Standard | Standard | **Extended** (custom attributes) |++**Key Insight**: OpenLIT's competitive advantage is **business intelligence** and **enhanced observability**.## Phase 2: Framework Analysis-### Step 2.1: Understand Framework Architecture+### Step 2.1: Explore Framework's Built-in Capabilities++**CRITICAL**: Before implementing, check if the framework has built-in tracing/monitoring:++```python+# Example: Discover OpenAI Agents has native tracing+import {framework}++# Look for tracing, monitoring, observability features+# Check documentation for:+# - Built-in tracing systems+# - Processor patterns+# - Event hooks+# - Monitoring callbacks++# Example: OpenAI Agents has TracingProcessor+from agents import TracingProcessor, set_trace_processors+```++**Framework Integration Decision Tree:**+1. **Native Integration Available** (e.g., OpenAI Agents TracingProcessor)+ - ✅ Use native integration for perfect hierarchy+ - ✅ Extends framework's built-in system+ - ✅ Better performance and reliability-**Key Questions:**-- What are the main execution flows?-- Which operations should be workflow-level vs component-level?-- Does the framework have built-in monitoring/tracing?-- What are the most important operations for users?+2. **Function Wrapping Required** (most frameworks)+ - ✅ Standard approach for frameworks without native tracing+ - ⚠️ Requires careful hierarchy management-**Framework Mapping:**+### Step 2.2: Map Framework Operations++**Workflow vs Component Operations:**```python# Example for any frameworkWORKFLOW_OPERATIONS = [- # High-level operations users care about- "pipeline.run",- "agent.execute",- "workflow.process"+ # High-level operations users care about in production+ "agent.run_sync", # OpenAI Agents+ "pipeline.run", # Haystack+ "workflow.execute" # Generic]COMPONENT_OPERATIONS = [- # Detailed operations for debugging+ # Detailed operations for debugging (detailed_tracing=True)"retriever.retrieve",- "generator.generate",- "embedder.embed"+ "generator.generate",+ "embedder.embed",+ "tool.call"]```-### Step 2.2: Test Built-in Framework Monitoring--Create test scripts to understand framework's native instrumentation:+### Step 2.3: Test Framework Methods and Patterns+**Method Discovery Script:**```python-# test_framework_native.py+# test_framework_exploration.pyimport {framework}-# Enable any built-in tracing/monitoring-# Run sample operations-# Document what spans/metrics are created natively-```-### Step 2.3: Identify OpenLIT Enhancement Opportunities+# Discover available methods and attributes+obj = SomeFrameworkClass()+print("Available methods:", [m for m in dir(obj) if not m.startswith('_')])-**Enhancement Areas:**-- [ ] Business metrics (cost, tokens, performance)-- [ ] Content capture (prompts, responses)-- [ ] Cross-system tracing (LLM provider integration)-- [ ] Advanced attributes (model details, usage patterns)-- [ ] Performance optimization+# Test built-in monitoring+if hasattr(obj, 'trace') or hasattr(obj, 'monitor'):+ print("Built-in tracing available!")++# Check for callback patterns+if hasattr(obj, 'add_callback') or hasattr(obj, 'on_event'):+ print("Event system available!")+```-## Phase 3: Implementation Structure+## Phase 3: Implementation Strategy-### Step 3.1: Create Directory Structure+### Step 3.1: Choose Integration Pattern+**Pattern 1: Native Integration (Preferred when available)**+```python+# Example: OpenAI Agents TracingProcessor+class OpenLITTracingProcessor(TracingProcessor):+ def on_trace_start(self, trace): pass+ def on_span_start(self, span): pass+ def on_span_end(self, span): pass```-sdk/python/src/openlit/instrumentation/{framework}/-├── __init__.py # Instrumentor class (70-100 lines)-├── {framework}.py # Sync wrapper (50-60 lines)-├── async_{framework}.py # Async wrapper (50-60 lines)-└── utils.py # Processing logic (200-400 lines)++**Pattern 2: Function Wrapping (Standard)**+```python+# Traditional OpenLIT pattern+wrap_function_wrapper("module", "Class.method", wrapper)```-### Step 3.2: Define Operation Mapping+### Step 3.2: Plan Span Naming Convention-In `utils.py`, create centralized operation mapping:+**CRITICAL**: Follow consistent naming pattern:+```python+# Standard format: "{operation_type} {operation_name}"+"agent Triage agent" # operation_type=agent, operation_name=Triage agent+"chat gpt-4o" # operation_type=chat, operation_name=gpt-4o+"retrieve documents" # operation_type=retrieve, operation_name=documents+"generate response" # operation_type=generate, operation_name=response+```+### Step 3.3: Plan Semantic Conventions Usage++**Extend semcov.py when needed:**```python-# Operation mapping for semantic conventions-OPERATION_MAP = {- "pipeline": "workflow",- "agent.execute": "agent",- "retriever.retrieve": "retrieve",- "generator.generate": "generate",- # Map framework operations to semantic operation types-}+# It's OK to add new semantic conventions for better attributes+class SemanticConvention:+ # Add framework-specific conventions+ GEN_AI_AGENT_DESCRIPTION = "gen_ai.agent.description"+ GEN_AI_AGENT_VERSION = "gen_ai.agent.version"+ GEN_AI_WORKFLOW_TYPE = "gen_ai.workflow.type"```-### Step 3.3: Plan Instrumentation Levels+## Phase 4: Code Implementation-**Workflow Level (Always Enabled):**-- High-level operations users monitor in production-- Minimal span count for performance-- Essential business metrics+### Step 4.1: Create Test Infrastructure First-**Component Level (detailed_tracing=True):**-- Detailed operations for debugging-- Comprehensive technical attributes-- Enhanced framework introspection+**Create span hierarchy test before implementing:**+```python+# test_span_hierarchy.py - Always create this first+import sys+sys.path.insert(0, 'sdk/python/src') # Use local source-## Phase 4: Code Implementation+import openlit+from opentelemetry.sdk.trace import TracerProvider+from opentelemetry.sdk.trace.export import SimpleSpanProcessor-### Step 4.1: Implement __init__.py+class CollectingSpanExporter:+ def __init__(self):+ self.spans = []++ def export(self, spans):+ for span in spans:+ self.spans.append({+ 'name': span.name,+ 'parent_id': format(span.parent.span_id, '016x') if span.parent else None,+ 'attributes': dict(span.attributes) if span.attributes else {}+ })+ return 0++def print_span_hierarchy(spans):+ """Print spans in hierarchical tree structure"""+ # Build parent-child relationships+ root_spans = [s for s in spans if s['parent_id'] is None]+ child_map = {}+ for span in spans:+ if span['parent_id']:+ child_map.setdefault(span['parent_id'], []).append(span)++ def print_span(span, level=0):+ indent = " " * level+ prefix = "├── " if level > 0 else ""+ print(f"{indent}{prefix}{span['name']}")++ span_id = format(int(span['parent_id'] or 0), '016x')+ for child in child_map.get(span_id, []):+ print_span(child, level + 1)++ for root in root_spans:+ print_span(root)++def test_framework():+ collector = CollectingSpanExporter()+ tracer_provider = TracerProvider()+ tracer_provider.add_span_processor(SimpleSpanProcessor(collector))++ openlit.init(detailed_tracing=True)++ # Test framework operations here+ # ...++ print_span_hierarchy(collector.spans)+```-Follow the optimized pattern:+### Step 4.2: Implement Based on Integration Pattern+**For Native Integration:**```python-"""-OpenLIT {Framework} Instrumentation - Optimized for Performance-"""--from typing import Collection-import importlib.metadata-from opentelemetry.instrumentation.instrumentor import BaseInstrumentor-from wrapt import wrap_function_wrapper--from openlit.instrumentation.{framework}.{framework} import general_wrap-from openlit.instrumentation.{framework}.async_{framework} import async_general_wrap--_instruments = ("{framework}-package >= X.Y.Z",)--class {Framework}Instrumentor(BaseInstrumentor):- """Optimized instrumentor for {Framework} with minimal overhead"""-- def instrumentation_dependencies(self) -> Collection[str]:- return _instruments-- def _instrument(self, **kwargs):- version = importlib.metadata.version("{framework}-package")- environment = kwargs.get("environment", "default")- application_name = kwargs.get("application_name", "default")- tracer = kwargs.get("tracer")- pricing_info = kwargs.get("pricing_info", {})- capture_message_content = kwargs.get("capture_message_content", False)- metrics = kwargs.get("metrics_dict")- disable_metrics = kwargs.get("disable_metrics")- detailed_tracing = kwargs.get("detailed_tracing", False)-- # Workflow operations (always enabled)- try:- wrap_function_wrapper(- "{framework}.module", "Class.method",- general_wrap("operation_type", version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics)- )- except Exception:- pass # Handle missing modules gracefully-- # Component operations (only if detailed_tracing enabled)- if detailed_tracing:- components = [- ("{framework}.module", "Component.method", "component_type"),- # List all component operations- ]-- for module, method, component_type in components:- try:- wrap_function_wrapper(- module, method,- general_wrap(component_type, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics)- )- except Exception:- pass # Each component wrapped individually-- def _uninstrument(self, **kwargs):- pass+# processor.py - When framework has built-in tracing+class OpenLITTracingProcessor(FrameworkProcessor):+ def on_span_start(self, span):+ # Create OpenTelemetry span with proper naming+ span_name = self._get_span_name(span) # {operation_type} {operation_name}+ otel_span = self._tracer.start_span(span_name, kind=SpanKind.CLIENT)++ # Set semantic convention attributes+ self._set_common_attributes(otel_span, span)++ def _get_span_name(self, span):+ operation_type = self._get_operation_type(span.span_data)+ operation_name = self._extract_operation_name(span.span_data)+ return f"{operation_type} {operation_name}"```-### Step 4.2: Implement Sync Wrapper+**For Function Wrapping:**+```python+# Follow existing 4-file structure+# __init__.py, sync_wrapper.py, async_wrapper.py, utils.py+```-In `{framework}.py`:+### Step 4.3: Implement Comprehensive Attributes+**Use semantic conventions extensively:**```python-"""-Sync wrapper for {Framework} operations-"""+def _set_span_attributes(self, span, data):+ # Standard framework attributes+ span.set_attribute(SemanticConvention.GEN_AI_SYSTEM, "framework_name")+ span.set_attribute(SemanticConvention.GEN_AI_OPERATION_NAME, operation_name)++ # Model information (critical for business intelligence)+ model = self._extract_model_info(data)+ if model:+ span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, model)++ # Content capture with MIME types (OpenLIT enhancement)+ if self._capture_message_content:+ self._capture_input_output_with_mime_types(span, data)++ # Business intelligence attributes+ self._capture_token_usage(span, data)+ self._capture_cost_metrics(span, data)++ # Framework-specific attributes+ if hasattr(data, 'agent_name'):+ span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, data.agent_name)+```++## Phase 5: Testing & Validation-from opentelemetry import context as context_api-from openlit.instrumentation.{framework}.utils import general_wrap_sync+### Step 5.1: Create Comprehensive Test Suite-def general_wrap(operation_type, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics):- """- General wrapper for {Framework} operations with optimized performance- """- def wrapper(wrapped, instance, args, kwargs):- # CRITICAL: Check instrumentation suppression- if context_api.get_value(context_api._SUPPRESS_INSTRUMENTATION_KEY):- return wrapped(*args, **kwargs)+**Test Categories:**+1. **Span Hierarchy Test** (most important)+2. **Competitive Comparison Test**+3. **Mock Framework Test** (when real framework unavailable)+4. **Error Resilience Test**- return general_wrap_sync(- wrapped, instance, args, kwargs,- operation_type, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics- )+**Span Hierarchy Validation:**+```python+def test_span_hierarchy():+ """Validate proper parent-child relationships and span naming"""+ spans = run_instrumentation_test()++ # Validate hierarchy structure+ assert len(spans) > 0, "Must generate spans"++ # Check root span exists+ root_spans = [s for s in spans if s['parent_id'] is None]+ assert len(root_spans) == 1, "Must have exactly one root span"- return wrapper+ # Validate span naming convention+ for span in spans:+ assert ' ' in span['name'], f"Span name '{span['name']}' should follow '{{operation_type}} {{operation_name}}' format"++ # Check semantic conventions+ for span in spans:+ attrs = span['attributes']+ assert 'gen_ai.system' in attrs, "Must have gen_ai.system attribute"```-### Step 4.3: Implement Async Wrapper--In `async_{framework}.py`:+### Step 5.2: Competitive Validation+**Critical Test - OpenLIT vs Competitors:**```python-"""-Async wrapper for {Framework} operations-"""--from opentelemetry import context as context_api-from openlit.instrumentation.{framework}.utils import general_wrap_async+def test_competitive_superiority():+ """Validate OpenLIT generates more/better spans than competitors"""++ # Test OpenLIT+ openlit_spans = test_openlit_instrumentation()++ # Test competitors (with error handling)+ try:+ competitor1_spans = test_competitor_instrumentation("openinference")+ competitor2_spans = test_competitor_instrumentation("openllmetry")+ except Exception as e:+ print(f"Competitor failed: {e}")+ competitor1_spans = competitor2_spans = 0++ # Validate OpenLIT superiority+ assert openlit_spans > 0, "OpenLIT must generate spans"+ print(f"OpenLIT: {openlit_spans} spans (with business intelligence)")+ print(f"Competitor 1: {competitor1_spans} spans")+ print(f"Competitor 2: {competitor2_spans} spans")++ # OpenLIT should provide comprehensive coverage+ assert openlit_spans >= max(competitor1_spans, competitor2_spans)+```-def async_general_wrap(operation_type, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics):- """- General async wrapper for {Framework} operations- """- def wrapper(wrapped, instance, args, kwargs):- # CRITICAL: Check instrumentation suppression- if context_api.get_value(context_api._SUPPRESS_INSTRUMENTATION_KEY):- return wrapped(*args, **kwargs)+### Step 5.3: Mock Testing When Framework Unavailable- return general_wrap_async(- wrapped, instance, args, kwargs,- operation_type, version, environment, application_name, tracer,- pricing_info, capture_message_content, metrics, disable_metrics- )+**Create realistic mock scenarios:**+```python+def test_with_mocks():+ """Test instrumentation when real framework isn't available"""+ tracer = trace.get_tracer(__name__)- return wrapper+ # Create realistic span hierarchy manually+ with tracer.start_as_current_span("agent Agent workflow") as root:+ root.set_attribute("gen_ai.agent.name", "Agent workflow")++ with tracer.start_as_current_span("agent Triage agent") as agent:+ agent.set_attribute("gen_ai.agent.name", "Triage agent")++ with tracer.start_as_current_span("chat gpt-4o") as chat:+ chat.set_attribute("gen_ai.request.model", "gpt-4o")```-### Step 4.4: Implement Utils.py+## Phase 6: Optimization-Core processing logic with comprehensive telemetry:+### Step 6.1: Performance Validation+**Measure instrumentation overhead:**```python-"""-Utility functions for {Framework} instrumentation-"""--import time-from opentelemetry import trace-from opentelemetry.trace import Status, StatusCode-from openlit.__helpers import (- common_framework_span_attributes,- handle_exception,- record_framework_metrics,- format_content-)-from openlit.semcov import SemanticConvention--# Operation mapping for semantic conventions-OPERATION_MAP = {- "pipeline": "workflow",- "component": "component",- # Add framework-specific mappings-}--def general_wrap_sync(wrapped, instance, args, kwargs, operation_type, version, environment,- application_name, tracer, pricing_info, capture_message_content,- metrics, disable_metrics):- """- Synchronous wrapper with comprehensive telemetry- """- # Extract operation details- gen_ai_endpoint = OPERATION_MAP.get(operation_type, operation_type)-- # Create span name following pattern: "{operation_type} {component/model}"- component_name = _extract_component_name(instance, operation_type)- span_name = f"{gen_ai_endpoint} {component_name}"-- with tracer.start_as_current_span(span_name) as span:- try:- # Set common framework attributes- _set_common_attributes(span, operation_type, instance, version,- environment, application_name, gen_ai_endpoint)-- # Record request details- _record_request_details(span, args, kwargs, capture_message_content)-- # Execute operation- start_time = time.time()- response = wrapped(*args, **kwargs)- end_time = time.time()-- # Process response- try:- _process_response(span, response, operation_type, capture_message_content)- span.set_status(Status(StatusCode.OK))- except Exception as e:- handle_exception(span, e)-- # Record metrics- if not disable_metrics and metrics:- _record_operation_metrics(metrics, operation_type, start_time, end_time,- environment, application_name)-- except Exception as e:- handle_exception(span, e)- raise-- return response--def _extract_component_name(instance, operation_type):- """Extract component name for span naming"""- if hasattr(instance, '__class__'):- return instance.__class__.__name__.lower()- return operation_type--def _set_common_attributes(span, operation_type, instance, version, environment,- application_name, gen_ai_endpoint):- """Set common framework span attributes"""- # Use centralized helper for standard attributes- common_framework_span_attributes(- span, "{framework}", None, None, environment,- application_name, version, gen_ai_endpoint, instance- )+def benchmark_performance():+ """Measure instrumentation performance impact"""+ import time- # Framework-specific attributes- span.set_attribute(SemanticConvention.GEN_AI_OPERATION_TYPE, operation_type)--def _record_request_details(span, args, kwargs, capture_message_content):- """Record request parameters and content"""- try:- # Extract and record input parameters- if capture_message_content:- # Record input content based on framework patterns- pass-- # Record technical parameters- _extract_technical_details(span, args, kwargs)-- except Exception:- pass # Don't fail instrumentation for attribute errors+ # Test without instrumentation+ start = time.time()+ run_framework_operations(count=100)+ baseline = time.time() - start++ # Test with instrumentation+ openlit.init()+ start = time.time()+ run_framework_operations(count=100)+ instrumented = time.time() - start++ overhead = ((instrumented - baseline) / baseline) * 100+ print(f"Instrumentation overhead: {overhead:.2f}%")+ assert overhead < 10, "Overhead should be under 10%"+```-def _process_response(span, response, operation_type, capture_message_content):- """Process and record response details"""- try:- if capture_message_content:- # Record output content- pass-- # Extract business metrics (tokens, cost, etc.)- _extract_business_metrics(span, response)-- # Extract technical framework details- _extract_framework_details(span, response, operation_type)-- except Exception:- pass # Don't fail instrumentation for response processing+### Step 6.2: Business Intelligence Validation-def _extract_technical_details(span, args, kwargs):- """Extract technical framework details for enhanced observability"""- # Component specifications- # Input/output types- # Framework-specific parameters- pass+**Verify OpenLIT's competitive advantages:**+```python+def test_business_intelligence():+ """Validate comprehensive business metrics capture"""+ spans = run_instrumentation_test()++ business_attributes = [+ 'gen_ai.usage.input_tokens',+ 'gen_ai.usage.output_tokens',+ 'gen_ai.client.operation.duration',+ 'gen_ai.request.model'+ ]++ for span in spans:+ attrs = span['attributes']+ found_attrs = [attr for attr in business_attributes if attr in attrs]+ print(f"Span '{span['name']}': {len(found_attrs)}/{len(business_attributes)} business attributes")+```-def _extract_business_metrics(span, response):- """Extract business intelligence metrics"""- # Token usage- # Cost calculations- # Performance metrics- pass+## Phase 7: Post-Change Cleanup-def _extract_framework_details(span, response, operation_type):- """Extract framework-specific technical details"""- # Use semantic conventions from semcov- # Component connections- # Pipeline metadata- # Execution details- pass+**CRITICAL**: Always run comprehensive cleanup after implementation:-def _record_operation_metrics(metrics, operation_type, start_time, end_time,- environment, application_name):- """Record framework operation metrics"""- record_framework_metrics(- metrics, "{framework}", None, None, environment,- application_name, start_time, end_time- )--# Async version-async def general_wrap_async(wrapped, instance, args, kwargs, operation_type, version,- environment, application_name, tracer, pricing_info,- capture_message_content, metrics, disable_metrics):- """- Asynchronous wrapper - mirrors sync implementation- """- # Same logic as sync version but with async/await- pass-```+### Step 7.1: Automated Code Quality Check-## Phase 5: Testing & Validation+```bash+#!/bin/bash+# cleanup_instrumentation.sh+FRAMEWORK_DIR="src/openlit/instrumentation/$1"-### Step 5.1: Create Test Scripts+echo "🧹 Cleaning up $FRAMEWORK_DIR..."-**Basic Functionality Test:**-```python-# test_basic_instrumentation.py-import openlit-from {framework} import *+# Remove trailing whitespace+find "$FRAMEWORK_DIR" -name "*.py" -exec sed -i '' 's/[[:space:]]*$//' {} \;-# Initialize with different configurations-openlit.init(detailed_tracing=False) # Workflow only-openlit.init(detailed_tracing=True) # Full instrumentation+# Add missing final newlines+find "$FRAMEWORK_DIR" -name "*.py" | while read file; do+ if [[ ! -s "$file" || $(tail -c1 "$file" | wc -l) -eq 0 ]]; then+ echo "" >> "$file"+ fi+done-# Test basic operations-# Verify span creation and attributes-```+# Check syntax+for file in "$FRAMEWORK_DIR"/*.py; do+ python3 -m py_compile "$file" || echo "❌ Syntax error in $file"+done-**Comparison Test:**-```python-# test_competitive_comparison.py-# Compare span count, attributes, and performance with competitors-# Document improvements and advantages+# Check line lengths+long_lines=$(find "$FRAMEWORK_DIR" -name "*.py" -exec awk 'length($0) > 80' {} \; | wc -l)+echo "Lines over 80 characters: $long_lines"```-### Step 5.2: Validation Checklist+### Step 7.2: Comprehensive Pylint Error Handling-- [ ] **Span Creation**: Correct number and hierarchy of spans-- [ ] **Span Naming**: Follows `{operation_type} {component}` pattern-- [ ] **Attributes**: Uses semantic conventions from semcov-- [ ] **Content Capture**: Captures input/output when enabled-- [ ] **Error Handling**: Graceful failure for missing components-- [ ] **Performance**: Minimal overhead impact-- [ ] **Cross-System**: Integrates with LLM provider instrumentations+**Run Pylint Check with Project Configuration:**+```bash+# Use the project's pylintrc configuration+cd sdk/python+pylint src/openlit/instrumentation/{framework}/ --rcfile=.pylintrc+```-## Phase 6: Optimization+**Note**: The project's `.pylintrc` disables many common warnings like `broad-exception-caught`, `too-many-locals`, etc.-### Step 6.1: Performance Optimization+### Step 7.3: Configure Project Pylint Rules-**Metrics to Track:**-- Line count reduction (target: 60-70% of original)-- Execution overhead (target: <5ms per operation)-- Memory usage impact-- Span creation efficiency+**Add `import-outside-toplevel` to disabled rules in `sdk/python/.pylintrc`:**-### Step 6.2: Code Efficiency+```ini+[MESSAGES CONTROL]+disable=+ # ... existing rules ...+ import-outside-toplevel # Essential for optional dependencies+```-**Optimization Patterns:**-- Use `general_wrap` pattern instead of individual wrappers-- Centralize operation mapping in `OPERATION_MAP`-- Leverage common helper functions-- Implement lazy loading for expensive operations+**Why**: Import-outside-toplevel warnings are triggered by correct optional dependency patterns in OpenLIT.-### Step 6.3: Competitive Validation Testing+### Step 7.4: Optimize Dummy Classes for Perfect Score-**Critical Testing Requirements:**-After implementation, create comprehensive competitive tests to validate superiority:+**Eliminate unnecessary dummy classes to achieve 10.00/10:**+#### **Use TYPE_CHECKING Pattern for Type-Only Classes**```python-# competitive_test.py - Test against OpenInference and OpenLLMetry-import openlit-import opentelemetry-from opentelemetry.sdk.trace import TracerProvider-from opentelemetry.sdk.trace.export import SimpleSpanProcessor-from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter--def test_openlit_vs_competitors():- """- Test OpenLIT against competitors for reliability and span generation- """-- # Test OpenLIT implementation- openlit.init(detailed_tracing=True)- spans_openlit = run_framework_operations()-- # Test competitor implementations (with error handling)- try:- # Configure competitor tracing- spans_competitor1 = test_competitor_implementation("openinference")- spans_competitor2 = test_competitor_implementation("openllmetry")- except Exception as e:- print(f"Competitor failed: {e}")- spans_competitor1 = spans_competitor2 = 0-- # Validate OpenLIT superiority- assert spans_openlit > 0, "OpenLIT must generate spans"- assert spans_openlit >= spans_competitor1, "OpenLIT should match/exceed competitor spans"- assert spans_openlit >= spans_competitor2, "OpenLIT should match/exceed competitor spans"+from typing import TYPE_CHECKING++try:+ from framework import MainClass+ if TYPE_CHECKING:+ from framework import TypeOnlyClass1, TypeOnlyClass2+ AVAILABLE = True+except ImportError:+ class MainClass:+ """Dummy class - actually used at runtime"""+ def method(self): return None- print(f"OpenLIT: {spans_openlit} spans")- print(f"Competitor 1: {spans_competitor1} spans")- print(f"Competitor 2: {spans_competitor2} spans")+ if TYPE_CHECKING:+ # Type hints only - don't exist at runtime+ TypeOnlyClass1 = Any+ TypeOnlyClass2 = Any+ AVAILABLE = False++# Use quoted type hints for methods+def process(self, item: "TypeOnlyClass1") -> None:```-**Validation Success Criteria:**-- OpenLIT generates spans when competitors fail-- OpenLIT span count >= competitor span count-- OpenLIT provides enhanced business intelligence-- OpenLIT demonstrates better error resilience-- OpenLIT shows superior OpenTelemetry SDK compatibility+#### **When R0903 is Still Acceptable**+- **Status**: ✅ **ACCEPTABLE** for classes actually used at runtime+- **Example**: TracingProcessor base class that's inherited from-### Step 6.4: Error Resilience Validation+**Target Score**: **10.00/10** is achievable for most instrumentations-**Version Compatibility Testing:**-```python-# Test across different OpenTelemetry SDK versions-def test_version_compatibility():- """Test instrumentation works across SDK versions"""- sdk_versions = ["1.19.0", "1.20.0", "1.21.0", "latest"]-- for version in sdk_versions:- try:- # Test with specific SDK version- test_result = run_instrumentation_test(sdk_version=version)- assert test_result.spans_generated > 0- print(f"✅ Compatible with OpenTelemetry SDK {version}")- except Exception as e:- print(f"❌ Failed with SDK {version}: {e}")-```--**Framework Version Testing:**+### Step 7.5: Common Pylint Issues and Fixes++#### **C0301: Line Too Long**+**Problem**: Lines exceed character limit (135 characters per project pylintrc)++**Fix Pattern**:```python-# Test across different framework versions-def test_framework_compatibility():- """Test instrumentation gracefully handles missing components"""-- # Test with minimal framework installation- # Test with full framework installation- # Test with missing optional components- # Verify graceful degradation in all cases-```+# BAD: Long function call+span.set_attribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, handle_not_given(kwargs.get("frequency_penalty"), 0.0))-## Phase 7: Post-Change Cleanup+# GOOD: Split across multiple lines+span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,+ handle_not_given(kwargs.get("frequency_penalty"), 0.0)+)-**CRITICAL**: After implementing any changes, always run comprehensive cleanup to ensure code quality compliance.+# BAD: Long function call with many parameters+common_span_attributes(scope, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT, SemanticConvention.GEN_AI_SYSTEM_OPENAI, server_address, server_port, request_model, response_model, environment, application_name, is_stream, tbt, ttft, version)++# GOOD: Split parameters logically+common_span_attributes(+ scope,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_OPENAI,+ server_address, server_port, request_model,+ response_model, environment, application_name,+ is_stream, tbt, ttft, version+)-### Step 7.1: Run Pylint Checks+# BAD: Long import+from openlit.instrumentation.framework.async_framework import async_general_wrap-```bash-# Check specific issues that commonly occur after changes-pylint --disable=all --enable=line-too-long,trailing-whitespace,missing-final-newline,trailing-newlines,syntax-error,unused-import,consider-iterating-dictionary,consider-using-in,too-many-nested-blocks src/openlit/instrumentation/{framework}/+# GOOD: Split import+from openlit.instrumentation.framework.async_framework import (+ async_general_wrap+)```-### Step 7.2: Common Issues and Automated Fixes--**1. Trailing Whitespace (C0303)**+#### **C0303: Trailing Whitespace**+**Automated Fix**:```bash-# Remove trailing whitespace from all files-for file in $(find src/openlit/instrumentation/{framework} -name "*.py"); do- sed 's/[[:space:]]*$//' "$file" > /tmp/temp_file && mv /tmp/temp_file "$file"-done+# Remove all trailing whitespace+find src/openlit/instrumentation/{framework} -name "*.py" -exec sed -i '' 's/[[:space:]]*$//' {} \;```-**2. Missing Final Newlines (C0304)**+#### **C0304: Missing Final Newline**+**Automated Fix**:```bash# Add missing final newlines-for file in $(find src/openlit/instrumentation/{framework} -name "*.py"); do+find src/openlit/instrumentation/{framework} -name "*.py" | while read file; doif [[ ! -s "$file" || $(tail -c1 "$file" | wc -l) -eq 0 ]]; thenecho "" >> "$file"fidone```-**3. Line Too Long (C0301)**-```bash-# Find long lines for manual fixing-awk 'length($0) > 80 { print FILENAME ":" NR ":" length($0) ":" $0 }' src/openlit/instrumentation/{framework}/*.py+#### **C0115: Missing Class Docstring**+**Fix Pattern**:+```python+# BAD: No docstring+class TracingProcessor:+ def force_flush(self): pass++# GOOD: Add descriptive docstring+class TracingProcessor:+ """Dummy TracingProcessor class for when agents is not available"""++ def force_flush(self):+ """Dummy force_flush method"""+ pass```-**Manual fixes for long lines:**+#### **C0116: Missing Function Docstring**+**Fix Pattern**:```python-# BAD: Long import-from openlit.instrumentation.framework.async_framework import async_general_wrap+# BAD: No docstring+def _extract_model_info(self, data):+ return data.model++# GOOD: Add descriptive docstring+def _extract_model_info(self, data):+ """Extract model information from span data or agent configuration"""+ return data.model+```-# GOOD: Split import-from openlit.instrumentation.framework.async_framework import (- async_general_wrap-)+#### **C0321: Multiple Statements on Single Line**+**Fix Pattern**:+```python+# BAD: Multiple statements+def force_flush(self): pass+class Trace: pass-# BAD: Long tuple-("very.long.module.name", "VeryLongClassName.very_long_method_name", "operation_type"),+# GOOD: Separate lines with proper formatting+def force_flush(self):+ """Dummy force_flush method"""+ pass-# GOOD: Split tuple-("very.long.module.name",- "VeryLongClassName.very_long_method_name", "operation_type"),+class Trace:+ """Dummy Trace class for when agents is not available"""+ pass+```-# BAD: Long function call-wrap_function_wrapper(module, method, general_wrap(operation_type, version, environment, application_name, tracer, pricing_info, capture_message_content, metrics, disable_metrics))--# GOOD: Split function call-wrap_function_wrapper(- module, method,- general_wrap(operation_type, version, environment, application_name,- tracer, pricing_info, capture_message_content,- metrics, disable_metrics)-)+#### **C0415: Import Outside Toplevel**+**Context**: These warnings are **ACCEPTABLE** and **REQUIRED** for optional dependencies++**Correct Pattern (DO NOT "FIX")**:+```python+# This is CORRECT for optional dependencies - keep as-is+def _instrument(self, **kwargs):+ try:+ from agents import set_trace_processors # CORRECT inside function+ set_trace_processors([processor])+ except ImportError:+ pass # Package not available```-**4. Syntax Errors (E0001)**-```bash-# Test compilation to catch syntax errors-python3 -m py_compile src/openlit/instrumentation/{framework}/*.py+**Why This is Correct**:+- Prevents import errors when target package isn't installed+- Essential for OpenLIT's optional dependency pattern+- Moving to top level would break instrumentation++**Only Fix When**:+```python+# ONLY move to top level for always-needed imports+import json # At top of file when always used+from typing import Any, Dict, Optional++def _capture_model_parameters(self, span, data):+ try:+ params = {"model": data.model}+ span.set_attribute("gen_ai.request.parameters", json.dumps(params))+ except Exception:+ pass```-**5. Code Quality Issues**+#### **E0602: Undefined Variable**+**Problem**: Variable used before definition-**Unused Imports (W0611)**+**Fix Pattern**:```python-# BAD: Unused import-from typing import Dict, Any, Optional # Optional not used+# BAD: Undefined variable+cost = get_chat_model_cost(model, pricing_info, input_tokens, output_tokens)-# GOOD: Remove unused-from typing import Dict, Any+# GOOD: Import the function+from openlit.__helpers import (+ common_framework_span_attributes,+ handle_exception,+ get_chat_model_cost # Add missing import+)```-**Consider Using 'in' (R1714)**+#### **Syntax Errors: Missing except/finally**+**Problem**: Try block without proper exception handling++**Fix Pattern**:```python-# BAD: Multiple or comparisons-if endpoint == "query" or endpoint == "query_async":+# BAD: Missing except block+def _extract_token_usage(self, span, data):+ try:+ usage = data.usage+ span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens)-# GOOD: Use 'in' operator-if endpoint in ("query", "query_async"):+# GOOD: Add except block+def _extract_token_usage(self, span, data):+ try:+ usage = data.usage+ span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens)+ except Exception:+ pass # Ignore errors in token usage extraction```-**Too Many Nested Blocks (R1702)**+#### **R1702: Too Many Nested Blocks**+**Fix Pattern**:```python# BAD: Deep nesting-if condition1:- if condition2:- if condition3:- if condition4:- if condition5:- # Too nested+def process_data(self, data):+ if hasattr(data, 'config'):+ if data.config:+ if hasattr(data.config, 'model'):+ if data.config.model:+ if isinstance(data.config.model, str):+ return data.config.model++# GOOD: Early returns+def process_data(self, data):+ if not hasattr(data, 'config'):+ return None+ if not data.config:+ return None+ if not hasattr(data.config, 'model'):+ return None+ if not data.config.model:+ return None+ if not isinstance(data.config.model, str):+ return None+ return data.config.model+```-# GOOD: Early returns or extraction-if not condition1:- return-if not condition2:- return-# Process main logic+#### **R0903: Too Few Public Methods**+**Context**: **ACCEPTABLE** and **EXPECTED** for dummy/placeholder classes++**Correct Pattern (DO NOT "FIX")**:+```python+# This is CORRECT for dummy classes - keep as-is+class TracingProcessor:+ """Dummy TracingProcessor class for when agents is not available"""++ def force_flush(self):+ """Dummy force_flush method"""+ return None++ def shutdown(self):+ """Dummy shutdown method"""+ return None```-### Step 7.3: Automated Cleanup Script+**Why This is Correct**:+- Dummy classes by design to prevent import errors+- Only need minimal methods to satisfy interfaces+- Adding unnecessary methods would be incorrect-Create a cleanup script for consistent application:+### Step 7.6: Automated Pylint Fix Script+**Create comprehensive fix script:**```bash#!/bin/bash-# cleanup_instrumentation.sh-+# fix_pylint_issues.shFRAMEWORK_DIR="src/openlit/instrumentation/$1"-echo "🧹 Cleaning up $FRAMEWORK_DIR..."+echo "🔧 Fixing Pylint issues in $FRAMEWORK_DIR..."# 1. Remove trailing whitespaceecho " ✂️ Removing trailing whitespace..."@@ -710,53 +765,239 @@ find "$FRAMEWORK_DIR" -name "*.py" | while read file; dofidone-# 3. Check for syntax errors+# 3. Check syntax errorsecho " 🔍 Checking syntax..."+syntax_errors=0for file in "$FRAMEWORK_DIR"/*.py; doif ! python3 -m py_compile "$file" 2>/dev/null; thenecho " ❌ Syntax error in $file"+ syntax_errors=$((syntax_errors + 1))fidone-# 4. Find long lines+# 4. Check line lengths and provide guidanceecho " 📏 Checking line lengths..."-long_lines=$(find "$FRAMEWORK_DIR" -name "*.py" -exec awk 'length($0) > 80 { print FILENAME ":" NR ":" length($0) }' {} \; | wc -l)-if [ "$long_lines" -gt 0 ]; then- echo " ⚠️ Found $long_lines lines over 80 characters"- find "$FRAMEWORK_DIR" -name "*.py" -exec awk 'length($0) > 80 { print FILENAME ":" NR ":" length($0) }' {} \;+long_lines=$(find "$FRAMEWORK_DIR" -name "*.py" -exec awk 'length($0) > 135 { print FILENAME ":" NR ":" length($0) ":" $0 }' {} \;)+if [ -n "$long_lines" ]; then+ echo " ⚠️ Long lines found (>135 chars):"+ echo "$long_lines" | head -5+ echo " 💡 Split long lines using the patterns in the guide"fi-echo "✅ Cleanup complete!"+# 5. Final validation+echo " ✅ Running final validation..."+if [ $syntax_errors -eq 0 ]; then+ echo "✅ All syntax errors fixed!"+else+ echo "❌ $syntax_errors syntax errors remaining - manual fix required"+fi++# 6. Run limited pylint check+echo " 🔍 Running key pylint checks..."+pylint --disable=all --enable=syntax-error,undefined-variable,trailing-whitespace,missing-final-newline "$FRAMEWORK_DIR" 2>/dev/null || echo "Some pylint issues remain - check manually"++echo "🎉 Pylint fix script completed!"+echo "💡 Run full pylint check and fix remaining issues manually"```-**Usage:**+### Step 7.7: Manual Fix Checklist++**Before committing, verify:**+- [ ] ✅ All files compile without syntax errors+- [ ] ✅ No trailing whitespace+- [ ] ✅ All files end with newline+- [ ] ✅ No lines over 135 characters (or split appropriately)+- [ ] ✅ All classes have docstrings+- [ ] ✅ All public methods have docstrings+- [ ] ✅ No undefined variables+- [ ] ✅ Try blocks have except/finally+- [ ] ✅ Imports are at top level when possible+- [ ] ✅ Nested blocks reduced where possible++### Step 7.8: Usage Instructions++**Run the fix script:**```bash-chmod +x cleanup_instrumentation.sh-./cleanup_instrumentation.sh llamaindex-./cleanup_instrumentation.sh haystack+chmod +x fix_pylint_issues.sh+./fix_pylint_issues.sh openai_agents+./fix_pylint_issues.sh haystack```-### Step 7.4: Final Validation+**Final validation:**+```bash+# Check specific framework+pylint src/openlit/instrumentation/openai_agents/-**Required Checks Before Commit:**-1. ✅ All files compile without syntax errors-2. ✅ No trailing whitespace-3. ✅ All files end with newline-4. ✅ No lines over 80 characters (or acceptable limit)-5. ✅ No unused imports-6. ✅ Code quality issues resolved+# Quick syntax check+python3 -m py_compile src/openlit/instrumentation/openai_agents/*.py-**Validation Commands:**-```bash-# Complete validation suite-python3 -m py_compile src/openlit/instrumentation/{framework}/*.py-grep -r " $" src/openlit/instrumentation/{framework}/ | wc -l # Should be 0-find src/openlit/instrumentation/{framework} -name "*.py" -exec awk 'length($0) > 80' {} \; | wc -l # Should be 0-pylint --disable=all --enable=line-too-long,trailing-whitespace,missing-final-newline,trailing-newlines,syntax-error,unused-import src/openlit/instrumentation/{framework}/-```--**Success Criteria:**-- Exit code 0 from py_compile (no syntax errors)-- 0 trailing whitespace instances-- 0 lines over length limit-- Clean pylint output for enabled checks\ No newline at end of file+# Test instrumentation still works+cd sdk/python && PYTHONPATH=src python -c "+import openlit+openlit.init()+print('✅ Instrumentation works after pylint fixes')+"+```++### Step 7.9: Testing After Pylint Fixes++**CRITICAL**: Always test that instrumentation works after fixing pylint issues:++```python+# test_after_pylint_fix.py+import sys+sys.path.insert(0, 'sdk/python/src')++import openlit+from opentelemetry.sdk.trace import TracerProvider+from opentelemetry.sdk.trace.export import SimpleSpanProcessor++class TestExporter:+ def __init__(self):+ self.spans = []+ def export(self, spans):+ self.spans.extend(spans)+ return 0+ def shutdown(self):+ pass++def test_instrumentation_after_fixes():+ """Test that instrumentation works after pylint fixes"""+ exporter = TestExporter()+ tracer_provider = TracerProvider()+ tracer_provider.add_span_processor(SimpleSpanProcessor(exporter))++ # Initialize OpenLIT+ openlit.init(detailed_tracing=True)++ # Run basic test based on framework+ # ... framework-specific test code ...++ tracer_provider.force_flush(1000)++ assert len(exporter.spans) > 0, "Instrumentation should generate spans"+ print(f"✅ Generated {len(exporter.spans)} spans after pylint fixes")++ # Validate span naming convention+ for span in exporter.spans:+ assert ' ' in span.name, f"Span '{span.name}' should follow '{{operation_type}} {{operation_name}}' format"++ print("✅ All tests passed - instrumentation working correctly")++if __name__ == "__main__":+ test_instrumentation_after_fixes()+```++## Code Standards & Patterns++### Span Naming Convention++**MANDATORY Format**: `{operation_type} {operation_name}`++```python+# ✅ CORRECT+"agent Triage agent"+"chat gpt-4o"+"retrieve documents"+"generate response"+"workflow multi-agent"++# ❌ INCORRECT+"Triage agent" # Missing operation type+"agent_execution" # Wrong format+"chat_completion_gpt4" # Wrong format+```++### Semantic Conventions Usage++**Extensive usage required - add to semcov.py when needed:**++```python+# Always use semantic conventions+span.set_attribute(SemanticConvention.GEN_AI_SYSTEM, "framework_name")+span.set_attribute(SemanticConvention.GEN_AI_OPERATION_NAME, operation_name)+span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, model)++# Add new conventions when needed+class SemanticConvention:+ GEN_AI_AGENT_DESCRIPTION = "gen_ai.agent.description"+ GEN_AI_WORKFLOW_TYPE = "gen_ai.workflow.type"+```++### Business Intelligence Attributes++**Always capture OpenLIT's competitive advantages:**++```python+# Token usage and cost tracking+span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)+span.set_attribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)+span.set_attribute(SemanticConvention.GEN_AI_CLIENT_OPERATION_DURATION, duration)++# Content capture with MIME types+span.set_attribute(SemanticConvention.GEN_AI_CONTENT_PROMPT, content)+span.set_attribute(SemanticConvention.GEN_AI_CONTENT_COMPLETION, response)+```++## Quality Checklist++### Phase Completion Checklist++**Phase 1 - Competitive Analysis:**+- [ ] Competitor repositories cloned and analyzed+- [ ] Integration patterns documented+- [ ] Competitive gaps identified+- [ ] OpenLIT advantages planned++**Phase 2 - Framework Analysis:**+- [ ] Built-in capabilities explored+- [ ] Method discovery completed+- [ ] Integration pattern chosen+- [ ] Operation mapping defined++**Phase 3 - Implementation Strategy:**+- [ ] Span naming convention defined+- [ ] Semantic conventions planned+- [ ] Test infrastructure designed++**Phase 4 - Implementation:**+- [ ] Test script created first+- [ ] Integration pattern implemented+- [ ] Semantic conventions used extensively+- [ ] Business intelligence captured++**Phase 5 - Testing:**+- [ ] Span hierarchy validated+- [ ] Competitive comparison passed+- [ ] Mock testing completed+- [ ] Error resilience tested++**Phase 6 - Optimization:**+- [ ] Performance overhead measured (<10%)+- [ ] Business intelligence validated+- [ ] Competitive advantages confirmed++**Phase 7 - Cleanup:**+- [ ] Code quality checks passed+- [ ] Syntax errors resolved+- [ ] Line length compliant+- [ ] Final validation completed++### Success Criteria++**Instrumentation is complete when:**+1. ✅ Generates more/better spans than competitors+2. ✅ Follows `{operation_type} {operation_name}` naming+3. ✅ Uses semantic conventions extensively+4. ✅ Captures comprehensive business intelligence+5. ✅ Maintains proper span hierarchy+6. ✅ Performance overhead <10%+7. ✅ Handles errors gracefully+8. ✅ **Perfect 10.00/10 pylint score achieved**++**OpenLIT Competitive Advantages Delivered:**+- 🎯 **Business Intelligence**: Cost tracking, token usage, performance metrics+- 🔗 **Cross-System Integration**: Links to LLM provider spans+- 🎨 **Enhanced Observability**: MIME types, structured content capture+- 🛡️ **Error Resilience**: Graceful degradation across framework versions+- 📊 **Comprehensive Coverage**: More spans and attributes than competitors\ No newline at end of filediff --git a/sdk/python/.pylintrc b/sdk/python/.pylintrcindex c670ade6..72ed5df0 100644--- a/sdk/python/.pylintrc+++ b/sdk/python/.pylintrc@@ -31,4 +31,5 @@ disable=expression-not-assigned,unnecessary-direct-lambda-call,no-member,- no-else-return\ No newline at end of file+ no-else-return,+ import-outside-toplevel\ No newline at end of filediff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.tomlindex 51eac815..695077ee 100644--- a/sdk/python/pyproject.toml+++ b/sdk/python/pyproject.toml@@ -1,6 +1,6 @@[tool.poetry]name = "openlit"-version = "1.34.28"+version = "1.34.29"description = "OpenTelemetry-native Auto instrumentation library for monitoring LLM Applications and GPUs, facilitating the integration of observability into your GenAI-driven projects"authors = ["OpenLIT"]license = "Apache-2.0"@@ -23,6 +23,7 @@ opentelemetry-exporter-otlp = "^1.30.0"opentelemetry-instrumentation = ">=0.52b0,<1.0.0"openai = "^1.1.1"anthropic = ">=0.42.0,<1.0.0"+openai-agents = ">=0.0.3"[build-system]requires = ["poetry-core>=1.1.0"]diff --git a/sdk/python/src/openlit/instrumentation/openai/utils.py b/sdk/python/src/openlit/instrumentation/openai/utils.pyindex af26e8fd..266817d6 100644--- a/sdk/python/src/openlit/instrumentation/openai/utils.py+++ b/sdk/python/src/openlit/instrumentation/openai/utils.py@@ -22,6 +22,14 @@ from openlit.__helpers import ()from openlit.semcov import SemanticConvention+def handle_not_given(value, default=None):+ """+ Handle OpenAI's NotGiven values by converting them to appropriate defaults.+ """+ if hasattr(value, '__class__') and value.__class__.__name__ == 'NotGiven':+ return default+ return value+def format_content(messages):"""Format the messages into a string for span events.@@ -248,9 +256,15 @@ def common_response_logic(scope, pricing_info, environment, application_name, meenvironment, application_name, is_stream, scope._tbt, scope._ttft, version)# Span Attributes for Request parameters specific to responses API- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, scope._kwargs.get("temperature", 1.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, scope._kwargs.get("top_p", 1.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, scope._kwargs.get("max_output_tokens", -1))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, handle_not_given(scope._kwargs.get("temperature"), 1.0))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_TOP_P,+ handle_not_given(scope._kwargs.get("top_p"), 1.0)+ )+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS,+ handle_not_given(scope._kwargs.get("max_output_tokens"), -1)+ )# Reasoning parametersreasoning = scope._kwargs.get("reasoning", {})@@ -427,20 +441,30 @@ def common_chat_logic(scope, pricing_info, environment, application_name, metriccost = get_chat_model_cost(request_model, pricing_info, input_tokens, output_tokens)# Common Span Attributes- common_span_attributes(scope,- SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT, SemanticConvention.GEN_AI_SYSTEM_OPENAI,- scope._server_address, scope._server_port, request_model, scope._response_model,- environment, application_name, is_stream, scope._tbt, scope._ttft, version)+ common_span_attributes(+ scope,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ SemanticConvention.GEN_AI_SYSTEM_OPENAI,+ scope._server_address, scope._server_port, request_model,+ scope._response_model, environment, application_name,+ is_stream, scope._tbt, scope._ttft, version+ )# Span Attributes for Request parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED, str(scope._kwargs.get("seed", "")))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, scope._kwargs.get("frequency_penalty", 0.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, scope._kwargs.get("max_tokens", -1))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY, scope._kwargs.get("presence_penalty", 0.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, scope._kwargs.get("stop", []))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, scope._kwargs.get("temperature", 1.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, scope._kwargs.get("top_p", 1.0))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_USER, scope._kwargs.get("user", ""))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_SEED, str(handle_not_given(scope._kwargs.get("seed"), "")))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY,+ handle_not_given(scope._kwargs.get("frequency_penalty"), 0.0)+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, handle_not_given(scope._kwargs.get("max_tokens"), -1))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY,+ handle_not_given(scope._kwargs.get("presence_penalty"), 0.0)+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, handle_not_given(scope._kwargs.get("stop"), []))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, handle_not_given(scope._kwargs.get("temperature"), 1.0))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, handle_not_given(scope._kwargs.get("top_p"), 1.0))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_USER, handle_not_given(scope._kwargs.get("user"), ""))# Span Attributes for Response parametersscope._span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, scope._response_id)@@ -571,8 +595,11 @@ def common_embedding_logic(scope, request_model, pricing_info, environment, applenvironment, application_name, False, scope._tbt, scope._ttft, version)# Span Attributes for Request parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_ENCODING_FORMATS, [scope._kwargs.get("encoding_format", "float")])- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_USER, scope._kwargs.get("user", ""))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_ENCODING_FORMATS,+ [handle_not_given(scope._kwargs.get("encoding_format"), "float")]+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_USER, handle_not_given(scope._kwargs.get("user"), ""))# Span Attributes for Cost and Tokensscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, scope._input_tokens)@@ -611,9 +638,12 @@ def common_image_logic(scope, request_model, pricing_info, environment, applicatenvironment, application_name, False, scope._tbt, scope._ttft, version)# Span Attributes for Request parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IMAGE_SIZE, scope._kwargs.get("size", "1024x1024"))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IMAGE_QUALITY, scope._kwargs.get("quality", "standard"))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_USER, scope._kwargs.get("user", ""))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_IMAGE_SIZE, handle_not_given(scope._kwargs.get("size"), "1024x1024"))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_IMAGE_QUALITY,+ handle_not_given(scope._kwargs.get("quality"), "standard")+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_USER, handle_not_given(scope._kwargs.get("user"), ""))# Extract response dataresponse_dict = scope._response_dict@@ -709,9 +739,12 @@ def common_audio_logic(scope, request_model, pricing_info, environment, applicatenvironment, application_name, False, scope._tbt, scope._ttft, version)# Span Attributes for Request parameters- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_AUDIO_VOICE, scope._kwargs.get("voice", "alloy"))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_AUDIO_RESPONSE_FORMAT, scope._kwargs.get("response_format", "mp3"))- scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_AUDIO_SPEED, scope._kwargs.get("speed", 1.0))+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_AUDIO_VOICE, handle_not_given(scope._kwargs.get("voice"), "alloy"))+ scope._span.set_attribute(+ SemanticConvention.GEN_AI_REQUEST_AUDIO_RESPONSE_FORMAT,+ handle_not_given(scope._kwargs.get("response_format"), "mp3")+ )+ scope._span.set_attribute(SemanticConvention.GEN_AI_REQUEST_AUDIO_SPEED, handle_not_given(scope._kwargs.get("speed"), 1.0))# Span Attributes for Costscope._span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)diff --git a/sdk/python/src/openlit/instrumentation/openai_agents/__init__.py b/sdk/python/src/openlit/instrumentation/openai_agents/__init__.pyindex fe58cc35..b99a5671 100644--- a/sdk/python/src/openlit/instrumentation/openai_agents/__init__.py+++ b/sdk/python/src/openlit/instrumentation/openai_agents/__init__.py@@ -1,42 +1,62 @@-"""Initializer of Auto Instrumentation of OpenAI Agents Functions"""+"""+OpenLIT OpenAI Agents Instrumentation - Native Tracing Integration+"""from typing import Collectionimport importlib.metadatafrom opentelemetry.instrumentation.instrumentor import BaseInstrumentor-from wrapt import wrap_function_wrapper-from openlit.instrumentation.openai_agents.openai_agents import (- create_agent-)+from openlit.instrumentation.openai_agents.processor import OpenLITTracingProcessor-_instruments = ('openai-agents >= 0.0.3',)+_instruments = ("openai-agents >= 0.0.3",)class OpenAIAgentsInstrumentor(BaseInstrumentor):- """- An instrumentor for openai-agents's client library.- """+ """OpenLIT instrumentor for OpenAI Agents using native tracing system"""def instrumentation_dependencies(self) -> Collection[str]:return _instrumentsdef _instrument(self, **kwargs):- application_name = kwargs.get('application_name', 'default')- environment = kwargs.get('environment', 'default')- tracer = kwargs.get('tracer')- event_provider = kwargs.get('event_provider')- metrics = kwargs.get('metrics_dict')- pricing_info = kwargs.get('pricing_info', {})- capture_message_content = kwargs.get('capture_message_content', False)- disable_metrics = kwargs.get('disable_metrics')- version = importlib.metadata.version('openai-agents')-- wrap_function_wrapper(- 'agents.agent',- 'Agent.__init__',- create_agent(version, environment, application_name,- tracer, event_provider, pricing_info, capture_message_content, metrics, disable_metrics),+ version = importlib.metadata.version("openai-agents")+ environment = kwargs.get("environment", "default")+ application_name = kwargs.get("application_name", "default")+ tracer = kwargs.get("tracer")+ pricing_info = kwargs.get("pricing_info", {})+ capture_message_content = kwargs.get("capture_message_content", False)+ metrics = kwargs.get("metrics_dict")+ disable_metrics = kwargs.get("disable_metrics")+ detailed_tracing = kwargs.get("detailed_tracing", False)++ # Create our processor with OpenLIT enhancements+ processor = OpenLITTracingProcessor(+ tracer=tracer,+ version=version,+ environment=environment,+ application_name=application_name,+ pricing_info=pricing_info,+ capture_message_content=capture_message_content,+ metrics=metrics,+ disable_metrics=disable_metrics,+ detailed_tracing=detailed_tracing)+ # Integrate with OpenAI Agents' native tracing system+ try:+ from agents import set_trace_processors+ # Replace existing processors with our enhanced processor+ set_trace_processors([processor])+ except ImportError:+ # Fallback: Add our processor to existing ones+ try:+ from agents import add_trace_processor+ add_trace_processor(processor)+ except ImportError:+ pass # Agents package may not have tracing+def _uninstrument(self, **kwargs):- # Proper uninstrumentation logic to revert patched methods- pass+ # Clear our processors+ try:+ from agents import set_trace_processors+ set_trace_processors([])+ except ImportError:+ passdiff --git a/sdk/python/src/openlit/instrumentation/openai_agents/openai_agents.py b/sdk/python/src/openlit/instrumentation/openai_agents/openai_agents.pydeleted file mode 100644index 7090bc75..00000000--- a/sdk/python/src/openlit/instrumentation/openai_agents/openai_agents.py+++ /dev/null@@ -1,65 +0,0 @@-"""-Module for monitoring AG2 API calls.-"""--import logging-from opentelemetry.trace import SpanKind, Status, StatusCode-from opentelemetry.sdk.resources import SERVICE_NAME, TELEMETRY_SDK_NAME, DEPLOYMENT_ENVIRONMENT-from openlit.__helpers import (- handle_exception,-)-from openlit.semcov import SemanticConvention--# Initialize logger for logging potential issues and operations-logger = logging.getLogger(__name__)--def set_span_attributes(span, version, operation_name, environment,- application_name, server_address, server_port, request_model):- """- Set common attributes for the span.- """-- # Set Span attributes (OTel Semconv)- span.set_attribute(TELEMETRY_SDK_NAME, 'openlit')- span.set_attribute(SemanticConvention.GEN_AI_OPERATION, operation_name)- span.set_attribute(SemanticConvention.GEN_AI_SYSTEM, SemanticConvention.GEN_AI_SYSTEM_AG2)- span.set_attribute(SemanticConvention.SERVER_ADDRESS, server_address)- span.set_attribute(SemanticConvention.SERVER_PORT, server_port)- span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, request_model)-- # Set Span attributes (Extras)- span.set_attribute(DEPLOYMENT_ENVIRONMENT, environment)- span.set_attribute(SERVICE_NAME, application_name)- span.set_attribute(SemanticConvention.GEN_AI_SDK_VERSION, version)--def create_agent(version, environment, application_name,- tracer, event_provider, pricing_info, capture_message_content, metrics, disable_metrics):- """- Generates a telemetry wrapper for GenAI function call- """- def wrapper(wrapped, instance, args, kwargs):- server_address, server_port = '127.0.0.1', 80-- agent_name = kwargs.get('name', 'openai_agent')- span_name = f'{SemanticConvention.GEN_AI_OPERATION_TYPE_CREATE_AGENT} {agent_name}'-- with tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:- try:- response = wrapped(*args, **kwargs)-- set_span_attributes(span, version, SemanticConvention.GEN_AI_OPERATION_TYPE_CREATE_AGENT,- environment, application_name, server_address, server_port, kwargs.get('model', 'gpt-4o'))- span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, agent_name)-- span.set_attribute(SemanticConvention.GEN_AI_AGENT_DESCRIPTION, kwargs.get('instructions', ''))-- span.set_status(Status(StatusCode.OK))-- return response-- except Exception as e:- handle_exception(span, e)- logger.error('Error in trace creation: %s', e)- return response-- return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/openai_agents/processor.py b/sdk/python/src/openlit/instrumentation/openai_agents/processor.pynew file mode 100644index 00000000..4d3236be--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/openai_agents/processor.py@@ -0,0 +1,600 @@+"""+OpenLIT OpenAI Agents Instrumentation - Native TracingProcessor Implementation+"""++import json+import time+from datetime import datetime+from typing import Any, Dict, Optional, TYPE_CHECKING++from opentelemetry import context as context_api+from opentelemetry.trace import SpanKind, Status, StatusCode, set_span_in_context+from opentelemetry.context import detach++from openlit.__helpers import (+ common_framework_span_attributes,+ handle_exception,+ record_framework_metrics,+ get_chat_model_cost+)+from openlit.semcov import SemanticConvention++# Try to import agents framework components with fallback+try:+ from agents import TracingProcessor+ if TYPE_CHECKING:+ from agents import Trace, Span+ TRACING_AVAILABLE = True+except ImportError:+ # Create dummy class for when agents is not available+ class TracingProcessor:+ """Dummy TracingProcessor class for when agents is not available"""++ def force_flush(self):+ """Dummy force_flush method"""+ return None++ def shutdown(self):+ """Dummy shutdown method"""+ return None++ if TYPE_CHECKING:+ # Type hints only - these don't exist at runtime when agents unavailable+ Trace = Any+ Span = Any++ TRACING_AVAILABLE = False+++class OpenLITTracingProcessor(TracingProcessor):+ """+ OpenLIT processor that integrates with OpenAI Agents' native tracing system+ Provides superior business intelligence while maintaining perfect hierarchy+ """++ def __init__(self, tracer: Any, version: str, environment: str,+ application_name: str, pricing_info: dict, capture_message_content: bool,+ metrics: Optional[Any], disable_metrics: bool, detailed_tracing: bool):+ if not TRACING_AVAILABLE:+ return++ self._tracer = tracer+ self._version = version+ self._environment = environment+ self._application_name = application_name+ self._pricing_info = pricing_info+ self._capture_message_content = capture_message_content+ self._metrics = metrics+ self._disable_metrics = disable_metrics+ self._detailed_tracing = detailed_tracing++ # Track spans for hierarchy+ self._root_spans: Dict[str, Any] = {}+ self._otel_spans: Dict[str, Any] = {}+ self._tokens: Dict[str, object] = {}+ self._span_start_times: Dict[str, float] = {}++ # Track handoff context for better span naming+ self._last_handoff_from: Optional[str] = None++ def on_trace_start(self, trace: "Trace") -> None:+ """Called when a trace is started - creates root workflow span"""+ if not TRACING_AVAILABLE:+ return++ # Create root workflow span with {operation_type} {operation_name} format+ workflow_name = getattr(trace, 'name', 'workflow')+ span_name = f"agent {workflow_name}" # Follow {operation_type} {operation_name} pattern++ # Use tracer.start_span for TracingProcessor pattern with proper context+ otel_span = self._tracer.start_span(+ name=span_name,+ kind=SpanKind.CLIENT+ )++ # Set common framework attributes for root span+ self._set_common_attributes(otel_span, trace.trace_id)++ # Set agent name for root span using semantic conventions+ if hasattr(trace, 'name') and trace.name:+ otel_span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, trace.name)++ # Set default model for root span+ otel_span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, "gpt-4o")++ self._root_spans[trace.trace_id] = otel_span+ self._span_start_times[trace.trace_id] = time.time()++ def on_span_start(self, span: "Span[Any]") -> None:+ """Called when a span is started - creates child spans with proper hierarchy"""+ if not TRACING_AVAILABLE or not hasattr(span, 'started_at') or not span.started_at:+ return++ start_time = self._parse_timestamp(span.started_at)++ # Determine parent span for proper hierarchy+ parent_span = None+ if span.parent_id and span.parent_id in self._otel_spans:+ parent_span = self._otel_spans[span.parent_id]+ elif span.trace_id in self._root_spans:+ parent_span = self._root_spans[span.trace_id]++ # Set context for parent-child relationship+ context = set_span_in_context(parent_span) if parent_span else None++ # Get semantic span name and operation type+ span_name = self._get_span_name(span)+ operation_type = self._get_operation_type(span.span_data)++ # Create span with proper context+ otel_span = self._tracer.start_span(+ name=span_name,+ context=context,+ start_time=self._as_utc_nano(start_time),+ kind=SpanKind.CLIENT+ )++ # Set common framework attributes for all spans+ self._set_common_framework_attributes(otel_span, operation_type)++ # Set span-specific attributes+ self._set_span_attributes(otel_span, span)++ # Track span and context+ self._otel_spans[span.span_id] = otel_span+ self._tokens[span.span_id] = context_api.attach(set_span_in_context(otel_span))+ self._span_start_times[span.span_id] = time.time()++ def on_span_end(self, span: "Span[Any]") -> None:+ """Called when a span is finished - adds business intelligence and ends span"""+ if not TRACING_AVAILABLE or span.span_id not in self._otel_spans:+ return++ otel_span = self._otel_spans[span.span_id]++ try:+ # Add response data and business intelligence+ self._process_span_completion(otel_span, span)++ # Set successful status+ otel_span.set_status(Status(StatusCode.OK))++ # Record metrics if enabled+ if not self._disable_metrics and self._metrics and span.span_id in self._span_start_times:+ start_time = self._span_start_times[span.span_id]+ end_time = time.time()+ operation_type = self._get_operation_type(span.span_data)+ record_framework_metrics(+ self._metrics, operation_type, SemanticConvention.GEN_AI_SYSTEM_OPENAI_AGENTS,+ "localhost", 80, self._environment, self._application_name,+ start_time, end_time+ )++ except Exception as e:+ handle_exception(otel_span, e)+ finally:+ # End span and cleanup+ otel_span.end()++ # Cleanup context+ if span.span_id in self._tokens:+ detach(self._tokens[span.span_id])+ del self._tokens[span.span_id]++ # Cleanup tracking+ if span.span_id in self._otel_spans:+ del self._otel_spans[span.span_id]+ if span.span_id in self._span_start_times:+ del self._span_start_times[span.span_id]++ def on_trace_end(self, trace: "Trace") -> None:+ """Called when a trace is finished - ends root span with business intelligence"""+ if not TRACING_AVAILABLE or trace.trace_id not in self._root_spans:+ return++ root_span = self._root_spans[trace.trace_id]++ try:+ # Add trace-level business intelligence+ self._process_trace_completion(root_span, trace)+ root_span.set_status(Status(StatusCode.OK))+ except Exception as e:+ handle_exception(root_span, e)+ finally:+ root_span.end()++ # Cleanup+ if trace.trace_id in self._root_spans:+ del self._root_spans[trace.trace_id]+ if trace.trace_id in self._span_start_times:+ del self._span_start_times[trace.trace_id]++ def _get_span_name(self, span: "Span[Any]") -> str:+ """Get semantic span name using {operation_type} {operation_name} format"""+ data = span.span_data+ operation_type = self._get_operation_type(data)++ # Extract operation name based on span type+ operation_name = "unknown"++ # Special handling for handoffs+ if hasattr(data, '__class__') and data.__class__.__name__ == 'HandoffSpanData':+ if hasattr(data, 'to_agent') and data.to_agent:+ operation_name = f"to {data.to_agent}"+ else:+ operation_name = "handoff"++ # Use agent name for agent spans+ elif hasattr(data, '__class__') and data.__class__.__name__ == 'AgentSpanData':+ # Try multiple possible attribute names for agent name+ agent_name = None++ for attr in ['agent_name', 'name', 'agent', 'agent_id']:+ if hasattr(data, attr):+ agent_name = getattr(data, attr)+ if agent_name and isinstance(agent_name, str):+ break++ # If still no agent name, try looking in context or other attributes+ if not agent_name:+ # Try context or other nested attributes+ if hasattr(data, 'context') and hasattr(data.context, 'agent'):+ agent_name = getattr(data.context.agent, 'name', None)+ elif hasattr(data, 'metadata') and hasattr(data.metadata, 'agent_name'):+ agent_name = data.metadata.agent_name++ if agent_name:+ operation_name = agent_name+ else:+ # If no agent name found, use a more descriptive fallback+ operation_name = "execution"++ # Use name if available for other spans+ elif hasattr(data, 'name') and isinstance(data.name, str):+ operation_name = data.name++ # Fallback to type-based names+ else:+ operation_name = getattr(data, 'type', 'operation')++ # Return formatted name: {operation_type} {operation_name}+ return f"{operation_type} {operation_name}"++ def _get_operation_type(self, data: Any) -> str:+ """Map span data to operation types"""+ class_name = data.__class__.__name__ if hasattr(data, '__class__') else str(type(data))++ mapping = {+ 'AgentSpanData': SemanticConvention.GEN_AI_OPERATION_TYPE_AGENT,+ 'GenerationSpanData': SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ 'FunctionSpanData': SemanticConvention.GEN_AI_OPERATION_TYPE_TOOLS,+ 'HandoffSpanData': SemanticConvention.GEN_AI_OPERATION_TYPE_AGENT,+ 'ResponseSpanData': SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT,+ }++ return mapping.get(class_name, SemanticConvention.GEN_AI_OPERATION_TYPE_FRAMEWORK)++ def _set_common_framework_attributes(self, span: Any, operation_type: str) -> None:+ """Set common framework attributes using semantic conventions"""+ # Create scope object for common_framework_span_attributes+ scope = type("GenericScope", (), {})()+ scope._span = span+ scope._start_time = time.time()+ scope._end_time = time.time()++ # Use common framework attributes helper+ # For framework operations, use localhost like other agent frameworks (AG2, Pydantic AI)+ common_framework_span_attributes(+ scope, SemanticConvention.GEN_AI_SYSTEM_OPENAI_AGENTS,+ "localhost", 80, self._environment, self._application_name,+ self._version, operation_type, None+ )++ def _set_common_attributes(self, span: Any, trace_id: str) -> None:+ """Set common framework attributes for root spans"""+ self._set_common_framework_attributes(span, SemanticConvention.GEN_AI_OPERATION_TYPE_FRAMEWORK)++ def _set_span_attributes(self, span: Any, agent_span: "Span[Any]") -> None:+ """Set span-specific attributes based on span data using semantic conventions"""+ data = agent_span.span_data++ # Agent-specific attributes using semantic conventions+ if hasattr(data, 'agent_name') and data.agent_name:+ span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, data.agent_name)+ elif hasattr(data, 'name') and data.name:+ span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, data.name)++ # Enhanced model information extraction+ model = self._extract_model_info(data, agent_span)+ if model:+ span.set_attribute(SemanticConvention.GEN_AI_REQUEST_MODEL, str(model))++ # Enhanced input/output capture with MIME types (OpenLIT enhancement)+ if self._capture_message_content:+ self._capture_input_output(span, data)++ # Enhanced token usage details (inspired by OpenInference)+ self._capture_detailed_token_usage(span, data)++ # Model invocation parameters as JSON (new feature from OpenInference)+ self._capture_model_parameters(span, data)++ # Tool/function information for tool calls+ if hasattr(data, '__class__') and 'Function' in data.__class__.__name__:+ if hasattr(data, 'function_name'):+ span.set_attribute(SemanticConvention.GEN_AI_TOOL_NAME, data.function_name)+ if hasattr(data, 'arguments'):+ span.set_attribute(SemanticConvention.GEN_AI_TOOL_ARGS, str(data.arguments))++ # Enhanced handoff information extraction+ if hasattr(data, '__class__') and 'Handoff' in data.__class__.__name__:+ target_agent = self._extract_handoff_target(data, agent_span)+ if target_agent:+ span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, target_agent)+ else:+ # Fallback for handoff spans without clear target+ span.set_attribute(SemanticConvention.GEN_AI_AGENT_NAME, "agent handoff")++ # Request/response IDs if available+ if hasattr(data, 'request_id'):+ span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, data.request_id)+ elif hasattr(data, 'response_id'):+ span.set_attribute(SemanticConvention.GEN_AI_RESPONSE_ID, data.response_id)++ def _extract_model_info(self, data: Any, agent_span: "Span[Any]") -> Optional[str]:+ """Extract model information from span data or agent configuration"""+ # Try direct model attributes first+ model_attrs = ['model', 'model_name', 'model_id', 'llm_model', 'openai_model']++ model = self._check_model_attrs(data, model_attrs)+ if model:+ return model++ # Try nested configuration objects+ config_attrs = ['config', 'configuration', 'client_config', 'llm_config']+ model = self._check_config_model_attrs(data, config_attrs, model_attrs)+ if model:+ return model++ # Try looking in the agent span itself+ if hasattr(agent_span, 'model'):+ return str(agent_span.model)++ # Try agent_config if available+ if hasattr(agent_span, 'agent_config'):+ model = self._check_model_attrs(agent_span.agent_config, model_attrs)+ if model:+ return model++ # Default fallback+ return "gpt-4o"++ def _check_model_attrs(self, obj: Any, model_attrs: list) -> Optional[str]:+ """Helper method to check model attributes on an object"""+ for attr in model_attrs:+ if not hasattr(obj, attr):+ continue+ model_value = getattr(obj, attr)+ if model_value and isinstance(model_value, str):+ return model_value+ return None++ def _check_config_model_attrs(self, data: Any, config_attrs: list, model_attrs: list) -> Optional[str]:+ """Helper method to check model attributes in nested configuration objects"""+ for config_attr in config_attrs:+ if not hasattr(data, config_attr):+ continue+ config = getattr(data, config_attr)+ if not config:+ continue+ model = self._check_model_attrs(config, model_attrs)+ if model:+ return model+ return None++ def _extract_handoff_target(self, data: Any, agent_span: "Span[Any]") -> Optional[str]:+ """Extract handoff target information with enhanced logic"""+ # Try direct target attributes+ target_attrs = ['to_agent', 'target_agent', 'destination_agent', 'next_agent']+ for attr in target_attrs:+ if hasattr(data, attr):+ target = getattr(data, attr)+ if target and isinstance(target, str):+ return f"to {target}"++ # Try from_agent for better handoff description+ from_attrs = ['from_agent', 'source_agent', 'previous_agent']+ for attr in from_attrs:+ if hasattr(data, attr):+ source = getattr(data, attr)+ if source and isinstance(source, str):+ return f"from {source}"++ # Try nested objects+ if hasattr(data, 'handoff_info'):+ info = data.handoff_info+ for attr in target_attrs + from_attrs:+ if hasattr(info, attr):+ value = getattr(info, attr)+ if value and isinstance(value, str):+ prefix = "to" if attr in target_attrs else "from"+ return f"{prefix} {value}"++ return None++ def _capture_input_output(self, span: Any, data: Any) -> None:+ """Capture input/output content with MIME type detection (OpenLIT enhancement)"""+ try:+ # Capture input content+ if hasattr(data, 'input') and data.input is not None:+ content = str(data.input)+ span.set_attribute(SemanticConvention.GEN_AI_CONTENT_PROMPT, content)+ # Set MIME type based on content structure+ if content.startswith('{') or content.startswith('['):+ span.set_attribute("gen_ai.content.prompt.mime_type", "application/json")+ else:+ span.set_attribute("gen_ai.content.prompt.mime_type", "text/plain")++ # Capture output/response content+ if hasattr(data, 'response') and data.response is not None:+ content = str(data.response)+ span.set_attribute(SemanticConvention.GEN_AI_CONTENT_COMPLETION, content)+ # Set MIME type based on content structure+ if content.startswith('{') or content.startswith('['):+ span.set_attribute("gen_ai.content.completion.mime_type", "application/json")+ else:+ span.set_attribute("gen_ai.content.completion.mime_type", "text/plain")++ except Exception:+ pass # Ignore export errors++ def _capture_detailed_token_usage(self, span: Any, data: Any) -> None:+ """Capture detailed token usage information (inspired by OpenInference)"""+ try:+ if hasattr(data, 'usage'):+ usage = data.usage++ # Standard token usage+ if hasattr(usage, 'input_tokens'):+ span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, usage.input_tokens)+ if hasattr(usage, 'output_tokens'):+ span.set_attribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, usage.output_tokens)++ # Enhanced token details (when available)+ if hasattr(usage, 'input_tokens_details'):+ details = usage.input_tokens_details+ if hasattr(details, 'cached_tokens'):+ span.set_attribute("gen_ai.usage.input_tokens.cached", details.cached_tokens)+ if hasattr(details, 'reasoning_tokens'):+ span.set_attribute("gen_ai.usage.input_tokens.reasoning", details.reasoning_tokens)++ if hasattr(usage, 'output_tokens_details'):+ details = usage.output_tokens_details+ if hasattr(details, 'reasoning_tokens'):+ span.set_attribute("gen_ai.usage.output_tokens.reasoning", details.reasoning_tokens)++ except Exception:+ pass # Ignore export errors++ def _capture_model_parameters(self, span: Any, data: Any) -> None:+ """Capture model invocation parameters as JSON (new feature from OpenInference)"""+ try:+ # Look for model configuration parameters+ params = {}++ # Common parameter attributes+ param_attrs = ['temperature', 'max_tokens', 'top_p', 'frequency_penalty', 'presence_penalty']+ for attr in param_attrs:+ if hasattr(data, attr):+ params[attr] = getattr(data, attr)++ # Try nested config objects+ if hasattr(data, 'config'):+ config = data.config+ for attr in param_attrs:+ if hasattr(config, attr):+ params[attr] = getattr(config, attr)++ # Try response object if available+ if hasattr(data, 'response') and hasattr(data.response, 'model_dump'):+ try:+ response_dict = data.response.model_dump()+ if response_dict and isinstance(response_dict, dict):+ # Extract model parameters from response+ if 'model' in response_dict:+ params['model'] = response_dict['model']+ if 'usage' in response_dict:+ params['usage'] = response_dict['usage']+ except Exception:+ pass++ # Set as JSON if we found any parameters+ if params:+ span.set_attribute("gen_ai.request.parameters", json.dumps(params))++ except Exception:+ pass # Ignore export errors++ def _process_span_completion(self, span: Any, agent_span: "Span[Any]") -> None:+ """Process span completion with enhanced business intelligence"""+ data = agent_span.span_data++ # Process response data if available+ self._process_response_data(span, data)++ # Extract and set token usage for business intelligence+ self._extract_token_usage(span, data)++ def _extract_token_usage(self, span: Any, data: Any) -> None:+ """Extract token usage and calculate costs (OpenLIT's business intelligence)"""+ try:+ # Try to extract token usage from various possible locations+ input_tokens = 0+ output_tokens = 0++ # Check direct usage attributes+ if hasattr(data, 'usage'):+ usage = data.usage+ input_tokens = getattr(usage, 'input_tokens', 0) or getattr(usage, 'prompt_tokens', 0)+ output_tokens = getattr(usage, 'output_tokens', 0) or getattr(usage, 'completion_tokens', 0)++ # Check response object+ elif hasattr(data, 'response') and hasattr(data.response, 'usage'):+ usage = data.response.usage+ input_tokens = getattr(usage, 'input_tokens', 0) or getattr(usage, 'prompt_tokens', 0)+ output_tokens = getattr(usage, 'output_tokens', 0) or getattr(usage, 'completion_tokens', 0)++ # Set token attributes+ span.set_attribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, input_tokens)+ span.set_attribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, output_tokens)+ span.set_attribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, input_tokens + output_tokens)++ # Calculate cost (OpenLIT's business intelligence advantage)+ model = getattr(data, 'model', 'gpt-4o')+ cost = get_chat_model_cost(model, self._pricing_info, input_tokens, output_tokens)+ span.set_attribute(SemanticConvention.GEN_AI_USAGE_COST, cost)++ except Exception:+ pass # Ignore errors in token usage extraction++ def _process_response_data(self, span: Any, data: Any) -> None:+ """Process response data with content capture"""+ if self._capture_message_content:+ self._capture_input_output(span, data)++ def _process_trace_completion(self, span: Any, trace: "Trace") -> None:+ """Process trace completion with business intelligence aggregation"""+ # Add trace-level metadata+ span.set_attribute(SemanticConvention.GEN_AI_OPERATION_NAME, "workflow")++ # Calculate total duration+ if trace.trace_id in self._span_start_times:+ start_time = self._span_start_times[trace.trace_id]+ duration = time.time() - start_time+ span.set_attribute(SemanticConvention.GEN_AI_CLIENT_OPERATION_DURATION, duration)++ def _parse_timestamp(self, timestamp: Any) -> float:+ """Parse timestamp from various formats"""+ if isinstance(timestamp, (int, float)):+ return float(timestamp)+ elif isinstance(timestamp, str):+ try:+ # Try parsing ISO format+ dt = datetime.fromisoformat(timestamp.replace('Z', '+00:00'))+ return dt.timestamp()+ except ValueError:+ return time.time()+ else:+ return time.time()++ def _as_utc_nano(self, timestamp: float) -> int:+ """Convert timestamp to UTC nanoseconds for OpenTelemetry"""+ return int(timestamp * 1_000_000_000)++ def force_flush(self) -> bool:+ """Force flush any pending spans (required by TracingProcessor)"""+ return True++ def shutdown(self) -> bool:+ """Shutdown the processor (required by TracingProcessor)"""+ return Truediff --git a/sdk/python/src/openlit/semcov/__init__.py b/sdk/python/src/openlit/semcov/__init__.pyindex 0c665264..574ffb74 100644--- a/sdk/python/src/openlit/semcov/__init__.py+++ b/sdk/python/src/openlit/semcov/__init__.py@@ -135,6 +135,7 @@ class SemanticConvention:GEN_AI_SYSTEM_FIRECRAWL = "firecrawl"GEN_AI_SYSTEM_LETTA = "letta"GEN_AI_SYSTEM_TOGETHER = "together"+ GEN_AI_SYSTEM_OPENAI_AGENTS = "openai_agents"GEN_AI_SYSTEM_PYDANTIC_AI = "pydantic_ai"# GenAI Framework Component Attributes (Standard)