Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
[Feat]: Refactor mem0 OpenTelemetry Instrumentation (#815)
Patcher149239e70a84
diff --git a/contributors/framework-instrumentation-guide.md b/contributors/framework-instrumentation-guide.mdindex 82ac1684..6463b5bd 100644--- a/contributors/framework-instrumentation-guide.md+++ b/contributors/framework-instrumentation-guide.md@@ -23,12 +23,24 @@ Before starting, ensure you have:- Access to competitor repositories for analysis- Understanding of OpenTelemetry concepts-**CRITICAL**: Always use local OpenLIT source code for testing:+**CRITICAL**: Always use the venv in the root directory and test with test.py:```bash-# DON'T install OpenLIT - use local source-PYTHONPATH=sdk/python/src python your_test_script.py+# ALWAYS use the venv in project root+source venv/bin/activate++# ALWAYS name your test file test.py (mandatory naming convention)+python test.py++# Alternative approach with explicit sys.path (for test.py):+sys.path.insert(0, "sdk/python/src")```+**MANDATORY Testing Standards:**+- Test file must be named `test.py` (not test_framework.py or similar)+- Always use `source venv/bin/activate` first+- Place test.py in project root directory+- Use sys.path.insert(0, "sdk/python/src") in test.py+## Phase 1: Competitive Analysis### Step 1.1: Clone and Research Competitor Implementations@@ -58,7 +70,8 @@ cd openllmetry/packages/openllmetry-instrumentation-{framework}**Deep Analysis Checklist:**- [ ] **Integration Pattern**: Function wrapping vs native integration (like TracingProcessor)- [ ] **Span Structure**: How many spans do they create and why?-- [ ] **Span Hierarchy**: Do they maintain proper parent-child relationships?+- [ ] **π¨ CRITICAL: Span Hierarchy**: Do they maintain proper parent-child relationships? (MOST IMPORTANT)+- [ ] **Threading Context**: How do they handle ThreadPoolExecutor and async contexts?- [ ] **Span Naming**: What naming convention do they use?- [ ] **Attributes**: What attributes do they capture? Check against semantic conventions- [ ] **Content Capture**: Do they capture input/output content with MIME types?@@ -67,6 +80,8 @@ cd openllmetry/packages/openllmetry-instrumentation-{framework}- [ ] **Performance**: How much overhead do they add?- [ ] **Coverage**: Which framework operations do they instrument?+**β οΈ SPAN HIERARCHY IS CRITICAL**: This is the #1 issue that breaks observability. Frameworks often use ThreadPoolExecutor or async operations that break OpenTelemetry context propagation, resulting in "all root spans" instead of proper parent-child relationships.+### Step 1.2: Document Competitive Gaps and OpenLIT AdvantagesCreate detailed comparison:@@ -106,35 +121,103 @@ framework_name/βββ __init__.py # Instrumentation setupβββ callback_handler.py # Framework's callback interface-# PATTERN 4: Standard Wrapper Pattern (CrewAI, Pydantic AI)-# Example: Most agent frameworks+# PATTERN 4: RECOMMENDED 4-File Structure (Mem0, CrewAI, Pydantic AI)+# Example: Complex frameworks needing performance optimizationframework_name/-βββ __init__.py # Instrumentation setup-βββ framework_name.py # Sync wrappers-βββ async_framework_name.py # Async wrappers-βββ utils.py # Shared utilities with context caching+βββ __init__.py # Instrumentation setup with separated SYNC/ASYNC methods+βββ framework_name.py # Sync wrappers with threading context fixes+βββ async_framework_name.py # Async wrappers+βββ utils.py # Shared utilities with context caching and __slots__++**π― PREFERRED STRUCTURE**: Use Pattern 4 for any framework with threading issues or performance needs.```-### Step 1.3: Performance Optimization Patterns+### Step 1.3: Threading Context Propagation Issues (CRITICAL)++**π¨ MOST COMMON FAILURE**: Frameworks using ThreadPoolExecutor break OpenTelemetry context propagation.++**Example Problem (like mem0):**+```python+# mem0/memory/main.py - Line 257-261+with concurrent.futures.ThreadPoolExecutor() as executor:+ future1 = executor.submit(self._add_to_vector_store, messages, ...)+ future2 = executor.submit(self._add_to_graph, messages, ...)+ # β This breaks context propagation - results in "all root spans"+```++**OpenLIT Solution Pattern:**+```python+# In utils.py - Threading context fix+def patch_concurrent_futures_context(span_context):+ """Patch ThreadPoolExecutor to propagate OpenTelemetry context."""+ original_submit = concurrent.futures.ThreadPoolExecutor.submit++ def patched_submit(self, fn, *args, **kwargs):+ # Capture current context and propagate it+ current_context = context.get_current()++ def context_wrapper(*args, **kwargs):+ token = context.attach(span_context)+ try:+ return fn(*args, **kwargs)+ finally:+ context.detach(token)++ return original_submit(self, context_wrapper, *args, **kwargs)++ concurrent.futures.ThreadPoolExecutor.submit = patched_submit+ return lambda: setattr(concurrent.futures.ThreadPoolExecutor, "submit", original_submit)++# In framework_name.py - Apply threading fix+def framework_wrap(gen_ai_endpoint, ...):+ def wrapper(wrapped, instance, args, kwargs):+ with tracer.start_as_current_span(gen_ai_endpoint, kind=span_kind, context=current_context) as span:+ span_context = set_span_in_context(span, context=current_context)++ # β Apply threading context fix for CLIENT operations+ if span_kind == SpanKind.CLIENT:+ restore_patch = patch_concurrent_futures_context(span_context)+ try:+ response = wrapped(*args, **kwargs)+ finally:+ restore_patch()+ else:+ response = wrapped(*args, **kwargs)+```++**π§ When to Apply Threading Fixes:**+- Framework uses `concurrent.futures.ThreadPoolExecutor`+- Framework uses `asyncio.run_in_executor`+- You see "all root spans" in your hierarchy test+- Child operations (OpenAI, Qdrant, etc.) appear as separate root spans++### Step 1.4: Performance Optimization Patterns**CRITICAL**: Study existing OpenLIT implementations for optimization patterns:```python# Study these proven patterns in existing instrumentations:+# - Mem0: threading context propagation, __slots__ optimization# - CrewAI: excellent agent patterns, lifecycle management# - OpenAI Agents: great tool handling (processor.py pattern)# - LangChain/LlamaIndex: mature framework patterns, utils.py caching# - Pydantic AI: context caching with PydanticAIInstrumentationContext-# Example: Context caching pattern from Pydantic AI+# Example: Optimized context caching with __slots__class FrameworkInstrumentationContext:"""Context object to cache expensive extractions."""+ __slots__ = ("instance", "args", "kwargs", "version", "environment",+ "application_name", "_agent_name", "_model_name", "_tools", "_messages")+def __init__(self, instance, args, kwargs, version, environment, application_name):self.instance = instanceself.args = argsself.kwargs = kwargs+ self.version = version+ self.environment = environment+ self.application_name = application_name# Cache expensive operations with lazy loadingself._agent_name = None@@ -297,65 +380,89 @@ class SemanticConvention:## Phase 4: Code Implementation-### Step 4.1: Create Test Infrastructure First+### Step 4.1: Create Span Hierarchy Test First (MANDATORY)++**π¨ CRITICAL**: Create test.py (mandatory name) with span hierarchy analysis BEFORE any implementation:-**Create span hierarchy test before implementing:**```python-# test_span_hierarchy.py - Always create this first+# test.py - MANDATORY filename, must be in project root+#!/usr/bin/env python3+"""+MANDATORY span hierarchy test for framework instrumentation.+This MUST be created first and MUST verify proper parent-child relationships.+"""+import sys-sys.path.insert(0, 'sdk/python/src') # Use local source+import os++sys.path.insert(0, "sdk/python/src") # MANDATORY path setupimport openlit-from opentelemetry.sdk.trace import TracerProvider-from opentelemetry.sdk.trace.export import SimpleSpanProcessor+import logging-class CollectingSpanExporter:- def __init__(self):- self.spans = []-- def export(self, spans):- for span in spans:- self.spans.append({- 'name': span.name,- 'parent_id': format(span.parent.span_id, '016x') if span.parent else None,- 'attributes': dict(span.attributes) if span.attributes else {}- })- return 0+# Minimal logging to focus on span hierarchy+logging.getLogger().setLevel(logging.ERROR)-def print_span_hierarchy(spans):- """Print spans in hierarchical tree structure"""- # Build parent-child relationships- root_spans = [s for s in spans if s['parent_id'] is None]- child_map = {}- for span in spans:- if span['parent_id']:- child_map.setdefault(span['parent_id'], []).append(span)-- def print_span(span, level=0):- indent = " " * level- prefix = "βββ " if level > 0 else ""- print(f"{indent}{prefix}{span['name']}")-- span_id = format(int(span['parent_id'] or 0), '016x')- for child in child_map.get(span_id, []):- print_span(child, level + 1)-- for root in root_spans:- print_span(root)+def test_framework_hierarchy():+ """+ π¨ CRITICAL TEST: Verify span hierarchy is correct, not "all root spans"+ """+ print("π§ Framework Instrumentation Test")+ print("=" * 40)-def test_framework():- collector = CollectingSpanExporter()- tracer_provider = TracerProvider()- tracer_provider.add_span_processor(SimpleSpanProcessor(collector))-+ # Initialize OpenLIT - use venv: source venv/bin/activate+ print("β Initializing OpenLIT...")openlit.init(detailed_tracing=True)-- # Test framework operations here- # ...-- print_span_hierarchy(collector.spans)++ # Import and use framework+ print("β Testing framework operations...")+ from your_framework import YourClass++ instance = YourClass()++ # Test operations that should create proper hierarchy+ result1 = instance.main_operation("test data", user_id="test_user")+ print(f"β Main operation result: {result1}")++ result2 = instance.secondary_operation("search query", user_id="test_user", limit=1)+ print(f"β Secondary operation result: {len(result2) if result2 else 0} items")++ print("\nπ― EXPECTED HIERARCHY:")+ print("πΈ ROOT: main_operation")+ print(" β³ internal_operation_1")+ print(" β³ openai_call (or similar)")+ print(" β³ database_operation")+ print("πΈ ROOT: secondary_operation")+ print(" β³ internal_operation_2")+ print(" β³ embedding_call")++ print("\nβ οΈ VERIFY: Check console output above for proper parent-child relationships")+ print("β If you see 'all root spans', you have threading context propagation issues!")++if __name__ == "__main__":+ test_framework_hierarchy()+```++**π§ SPAN HIERARCHY DEBUGGING COMMANDS:**++```bash+# Run test.py and analyze hierarchy+source venv/bin/activate # MANDATORY first step+python test.py > spans.json 2>&1++# Parse and analyze hierarchy (create this script)+python3 -c "+import json+# ... hierarchy parsing script from mem0 experience ...+# Shows proper tree structure or identifies 'all root spans' issue+"```+**β οΈ HIERARCHY TESTING IS THE MOST IMPORTANT STEP**:+- If hierarchy is broken (all root spans), nothing else matters+- This is the #1 failure mode in framework instrumentation+- Threading context propagation issues MUST be caught early+### Step 4.2: Implement Based on Integration Pattern**For Native Integration:**@@ -689,16 +796,27 @@ echo "Lines over 80 characters: $long_lines"### Step 7.2: Comprehensive Pylint Error Handling+**π¨ MANDATORY: 10.00/10 PYLINT SCORE REQUIRED**+**Run Pylint Check with Project Configuration:**```bash-# Use the project's pylintrc configuration+# ALWAYS use venv first, then run from SDK directory+source venv/bin/activatecd sdk/python-pylint src/openlit/instrumentation/{framework}/ --rcfile=.pylintrc+python -m pylint src/openlit/instrumentation/{framework}/ --rcfile=.pylintrc++# TARGET SCORE: 10.00/10 (not 9.5, must be perfect)+```-# Target scores:-# - Individual files: 9.5-10.0/10-# - Module overall: 9.0-9.5/10-# - Perfect 10.0/10 is achievable by ignoring import errors+**Run Script for Perfect 10.0/10:**+```bash+# Use the script pattern from mem0 success+#!/bin/bash+cd /Users/user/openlit # Adjust path+source venv/bin/activate+export PYTHONPATH="/Users/user/openlit/sdk/python/src:$PYTHONPATH"+cd sdk/python+python -m pylint src/openlit/instrumentation/{framework}/ --rcfile=.pylintrc```**Note**: The project's `.pylintrc` disables many common warnings like `broad-exception-caught`, `too-many-locals`, etc.@@ -1276,16 +1394,22 @@ span.set_attribute(SemanticConvention.GEN_AI_CONTENT_COMPLETION, response)### Success Criteria**Instrumentation is complete when:**-1. β **Competitive Analysis**: Studied OpenInference, OpenLLMetry, AgentOps, LangFuse-2. β **Target SDK Study**: Cloned and analyzed framework's internal structure-3. β **Semantic Conventions**: Uses SemanticConvention extensively (no hardcoded strings)-4. β **Span Hierarchy**: Generates more/better spans than competitors-5. β **Naming Convention**: Follows `{operation_type} {operation_name}` format-6. β **Context Caching**: Implements performance optimization patterns-7. β **Business Intelligence**: Captures comprehensive cost/token/performance data-8. β **Performance**: Overhead <10% with benchmarking vs competitors-9. β **Error Resilience**: Handles framework version differences gracefully-10. β **Code Quality**: Achieves 9.5-10.0/10 pylint score+1. π¨ **CRITICAL: Proper Span Hierarchy**: NO "all root spans" - proper parent-child relationships maintained+2. π¨ **CRITICAL: Threading Context Fixes**: Handles ThreadPoolExecutor and async context propagation+3. β **Perfect Pylint Score**: Achieves exactly 10.00/10 (not 9.5, must be perfect)+4. β **Mandatory test.py**: Uses exactly "test.py" filename with venv testing+5. β **Competitive Analysis**: Studied OpenInference, OpenLLMetry, AgentOps, LangFuse+6. β **Target SDK Study**: Cloned and analyzed framework's internal structure+7. β **Semantic Conventions**: Uses SemanticConvention extensively (no hardcoded strings)+8. β **Superior Span Coverage**: Generates more/better spans than competitors+9. β **Naming Convention**: Follows `{operation_type} {operation_name}` format+10. β **4-File Structure**: Uses recommended Pattern 4 when needed for complex frameworks+11. β **Context Caching**: Implements performance optimization patterns with __slots__+12. β **Business Intelligence**: Captures comprehensive cost/token/performance data+13. β **Performance**: Overhead <10% with benchmarking vs competitors+14. β **Error Resilience**: Handles framework version differences gracefully++**β οΈ PRIORITY ORDER**: Span hierarchy (#1) and threading fixes (#2) are MOST CRITICAL - everything else is secondary.**OpenLIT Competitive Advantages Delivered:**- π― **Superior Business Intelligence**: Detailed cost tracking, token usage, performance metricsdiff --git a/otel-gpu-collector/collector.py b/otel-gpu-collector/collector.pyindex 3a17761b..03c21c06 100644--- a/otel-gpu-collector/collector.py+++ b/otel-gpu-collector/collector.py@@ -6,51 +6,57 @@ import jsonimport signalimport time+# Create a JSON loggerclass JSONFormatter(logging.Formatter):def format(self, record):log_record = {"level": record.levelname,"message": record.getMessage(),- "timestamp": time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(record.created)),+ "timestamp": time.strftime(+ "%Y-%m-%d %H:%M:%S", time.localtime(record.created)+ ),"name": record.name,"application_name": record.application_name,- "environment": record.environment+ "environment": record.environment,}if record.exc_info:log_record["exception"] = self.formatException(record.exc_info)return json.dumps(log_record)+# Custom filter to add fixed information to log recordsclass ContextFilter(logging.Filter):def __init__(self, application_name, environment):super().__init__()self.application_name = application_nameself.environment = environment-+def filter(self, record):record.application_name = self.application_namerecord.environment = self.environmentreturn True+def setup_logger(application_name, environment):logger = logging.getLogger("otel-gpu-collector")logger.setLevel(logging.INFO)-+# Log to stdouthandler = logging.StreamHandler(sys.stdout)handler.setFormatter(JSONFormatter())-+logger.addHandler(handler)-+# Add context filter to loggerlogger.addFilter(ContextFilter(application_name, environment))-+return logger+# Load environment variables-application_name = os.getenv('GPU_APPLICATION_NAME', 'default')-environment = os.getenv('GPU_ENVIRONMENT', 'default')+application_name = os.getenv("GPU_APPLICATION_NAME", "default")+environment = os.getenv("GPU_ENVIRONMENT", "default")# Initialize loggerlogger = setup_logger(application_name, environment)@@ -58,30 +64,38 @@ logger = setup_logger(application_name, environment)# Flag to control the infinite loopkeep_running = True+def signal_handler(sig, frame):global keep_runninglogger.info("Received termination signal", extra={"signal": sig})keep_running = False+def main():logger.info("Starting otel-gpu-collector")try:# Log environment variableslogger.info("Environment variables loaded")-+# Initialize OpenLit with provided parameterslogger.info("Initializing GPU OpenTelemetry Instrumentation library OpenLIT")- openlit.init(collect_gpu_stats=True, application_name=application_name, environment=environment)+ openlit.init(+ collect_gpu_stats=True,+ application_name=application_name,+ environment=environment,+ )logger.info("Initialization complete")-+# Register signal handlerssignal.signal(signal.SIGTERM, signal_handler)signal.signal(signal.SIGINT, signal_handler)-+# Log that signal handlers are registered- logger.info("Signal handlers registered", extra={"signals": ["SIGTERM", "SIGINT"]})-+ logger.info(+ "Signal handlers registered", extra={"signals": ["SIGTERM", "SIGINT"]}+ )+# Keep the script running indefinitelyglobal keep_runningwhile keep_running:@@ -92,5 +106,6 @@ def main():finally:logger.info("Shutting down GPU Monitor Service")+if __name__ == "__main__":- main()\ No newline at end of file+ main()diff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.tomlindex 77a3e46e..a1699c4a 100644--- a/sdk/python/pyproject.toml+++ b/sdk/python/pyproject.toml@@ -1,6 +1,6 @@[tool.poetry]name = "openlit"-version = "1.34.33"+version = "1.34.34"description = "OpenTelemetry-native Auto instrumentation library for monitoring LLM Applications and GPUs, facilitating the integration of observability into your GenAI-driven projects"authors = ["OpenLIT"]license = "Apache-2.0"diff --git a/sdk/python/src/openlit/instrumentation/mem0/__init__.py b/sdk/python/src/openlit/instrumentation/mem0/__init__.pyindex 32e94a2e..369cbae9 100644--- a/sdk/python/src/openlit/instrumentation/mem0/__init__.py+++ b/sdk/python/src/openlit/instrumentation/mem0/__init__.py@@ -1,5 +1,7 @@# pylint: disable=useless-return, bad-staticmethod-argument, disable=duplicate-code-"""Initializer of Auto Instrumentation of mem0 Functions"""+"""+Optimized Auto Instrumentation of mem0 Functions following OpenLIT Framework Guide.+"""from typing import Collectionimport importlib.metadata@@ -7,82 +9,297 @@ from opentelemetry.instrumentation.instrumentor import BaseInstrumentorfrom wrapt import wrap_function_wrapperfrom openlit.instrumentation.mem0.mem0 import mem0_wrap+from openlit.instrumentation.mem0.async_mem0 import async_mem0_wrap_instruments = ("mem0ai >= 0.1.32",)-WRAPPED_METHODS = [+# Optimized method configuration with performance priorities+SYNC_METHODS = [+ # Memory initialization+ {+ "package": "mem0",+ "object": "Memory.__init__",+ "endpoint": "memory init",+ "priority": "critical",+ },+ # Core Memory operations{"package": "mem0","object": "Memory.add",- "endpoint": "mem0.memory_add",- "wrapper": mem0_wrap,+ "endpoint": "memory add",+ "priority": "high",},{"package": "mem0",- "object": "Memory.get_all",- "endpoint": "mem0.memory_get_all",- "wrapper": mem0_wrap,+ "object": "Memory.search",+ "endpoint": "memory search",+ "priority": "high",},{"package": "mem0","object": "Memory.get",- "endpoint": "mem0.memory_get",- "wrapper": mem0_wrap,+ "endpoint": "memory get",+ "priority": "medium",},{"package": "mem0",- "object": "Memory.search",- "endpoint": "mem0.memory_search",- "wrapper": mem0_wrap,+ "object": "Memory.get_all",+ "endpoint": "memory get_all",+ "priority": "medium",},{"package": "mem0","object": "Memory.update",- "endpoint": "mem0.memory_update",- "wrapper": mem0_wrap,+ "endpoint": "memory update",+ "priority": "medium",},{"package": "mem0",- "object": "Memory.update",- "endpoint": "mem0.memory_update",- "wrapper": mem0_wrap,+ "object": "Memory.delete",+ "endpoint": "memory delete",+ "priority": "medium",+ },+ {+ "package": "mem0",+ "object": "Memory.delete_all",+ "endpoint": "memory delete_all",+ "priority": "low",+ },+ {+ "package": "mem0",+ "object": "Memory.history",+ "endpoint": "memory history",+ "priority": "low",+ },+ {+ "package": "mem0",+ "object": "Memory.reset",+ "endpoint": "memory reset",+ "priority": "low",+ },+ {+ "package": "mem0",+ "object": "Memory.chat",+ "endpoint": "memory chat",+ "priority": "medium",+ },+ # Internal operations for detailed tracing+ {+ "package": "mem0.memory.main",+ "object": "Memory._add_to_vector_store",+ "endpoint": "memory add_to_vector_store",+ "priority": "medium",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._add_to_graph",+ "endpoint": "memory add_to_graph",+ "priority": "medium",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._search_vector_store",+ "endpoint": "memory search_vector_store",+ "priority": "medium",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._get_all_from_vector_store",+ "endpoint": "memory get_all_from_vector_store",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._create_memory",+ "endpoint": "memory create_memory",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._update_memory",+ "endpoint": "memory update_memory",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._delete_memory",+ "endpoint": "memory delete_memory",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "Memory._create_procedural_memory",+ "endpoint": "memory create_procedural_memory",+ "priority": "low",+ },+]++ASYNC_METHODS = [+ # AsyncMemory initialization+ {+ "package": "mem0",+ "object": "AsyncMemory.__init__",+ "endpoint": "memory init",+ "priority": "critical",+ },+ # Core AsyncMemory operations+ {+ "package": "mem0",+ "object": "AsyncMemory.add",+ "endpoint": "memory add",+ "priority": "high",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.search",+ "endpoint": "memory search",+ "priority": "high",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.get",+ "endpoint": "memory get",+ "priority": "medium",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.get_all",+ "endpoint": "memory get_all",+ "priority": "medium",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.update",+ "endpoint": "memory update",+ "priority": "medium",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.delete",+ "endpoint": "memory delete",+ "priority": "medium",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.delete_all",+ "endpoint": "memory delete_all",+ "priority": "low",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.history",+ "endpoint": "memory history",+ "priority": "low",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.reset",+ "endpoint": "memory reset",+ "priority": "low",+ },+ {+ "package": "mem0",+ "object": "AsyncMemory.chat",+ "endpoint": "memory chat",+ "priority": "medium",+ },+ # Internal async operations for detailed tracing+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._add_to_graph",+ "endpoint": "memory add_to_graph",+ "priority": "medium",+ },+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._search_vector_store",+ "endpoint": "memory search_vector_store",+ "priority": "medium",+ },+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._get_all_from_vector_store",+ "endpoint": "memory get_all_from_vector_store",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._create_memory",+ "endpoint": "memory create_memory",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._update_memory",+ "endpoint": "memory update_memory",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._delete_memory",+ "endpoint": "memory delete_memory",+ "priority": "low",+ },+ {+ "package": "mem0.memory.main",+ "object": "AsyncMemory._create_procedural_memory",+ "endpoint": "memory create_procedural_memory",+ "priority": "low",},]class Mem0Instrumentor(BaseInstrumentor):- """An instrumentor for mem0's client library."""+ """Optimized instrumentor for mem0's client library."""def instrumentation_dependencies(self) -> Collection[str]:return _instrumentsdef _instrument(self, **kwargs):+ """Optimized instrumentation with performance considerations."""+ # Extract configurationapplication_name = kwargs.get("application_name")environment = kwargs.get("environment")tracer = kwargs.get("tracer")pricing_info = kwargs.get("pricing_info")capture_message_content = kwargs.get("capture_message_content")- version = importlib.metadata.version("mem0ai")- for wrapped_method in WRAPPED_METHODS:- wrap_package = wrapped_method.get("package")- wrap_object = wrapped_method.get("object")- gen_ai_endpoint = wrapped_method.get("endpoint")- wrapper = wrapped_method.get("wrapper")- wrap_function_wrapper(- wrap_package,- wrap_object,- wrapper(- gen_ai_endpoint,- version,- environment,- application_name,- tracer,- pricing_info,- capture_message_content,- ),- )+ # Cache version lookup+ try:+ version = importlib.metadata.version("mem0ai")+ except importlib.metadata.PackageNotFoundError:+ version = "unknown"++ # Pre-create wrapper args+ wrapper_args = (+ version,+ environment,+ application_name,+ tracer,+ pricing_info,+ capture_message_content,+ )++ # Instrument sync methods+ self._wrap_methods(SYNC_METHODS, mem0_wrap, wrapper_args)++ # Instrument async methods+ self._wrap_methods(ASYNC_METHODS, async_mem0_wrap, wrapper_args)++ def _wrap_methods(self, methods, wrapper_func, wrapper_args):+ """Efficiently wrap methods with error handling."""+ for method_config in methods:+ try:+ wrap_function_wrapper(+ method_config["package"],+ method_config["object"],+ wrapper_func(method_config["endpoint"], *wrapper_args),+ )+ except (ImportError, AttributeError):+ # Gracefully handle missing methods in different mem0 versions+ pass@staticmethoddef _uninstrument(self, **kwargs):- pass+ """Uninstrument mem0."""+ # Currently no cleanup neededdiff --git a/sdk/python/src/openlit/instrumentation/mem0/async_mem0.py b/sdk/python/src/openlit/instrumentation/mem0/async_mem0.pynew file mode 100644index 00000000..214f3130--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/mem0/async_mem0.py@@ -0,0 +1,105 @@+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+"""+Optimized async mem0 instrumentation following OpenLIT Framework Guide.+"""++import time+from opentelemetry.trace import SpanKind, Status, StatusCode, set_span_in_context+from opentelemetry import context+from opentelemetry.instrumentation.utils import _SUPPRESS_INSTRUMENTATION_KEY+from openlit.__helpers import handle_exception+from openlit.instrumentation.mem0.utils import (+ Mem0Context,+ set_mem0_span_attributes,+ set_mem0_content_attributes,+ logger,+)+++def async_mem0_wrap(+ gen_ai_endpoint,+ version,+ environment,+ application_name,+ tracer,+ pricing_info,+ capture_message_content,+):+ """+ Optimized wrapper for async mem0 operations with proper hierarchy and performance.++ Returns:+ Wrapper function for async mem0 operations+ """++ def wrapper(wrapped, instance, args, kwargs):+ """+ Inner wrapper with comprehensive tracing for async operations.+ """+ # Suppression check+ if context.get_value(_SUPPRESS_INSTRUMENTATION_KEY):+ return wrapped(*args, **kwargs)++ # Handle AsyncMemory.__init__ to create parent span for initialization operations+ if gen_ai_endpoint == "memory init":+ with tracer.start_as_current_span("memory init", kind=SpanKind.INTERNAL):+ return wrapped(*args, **kwargs)++ # Determine span type efficiently+ is_internal = any(+ marker in gen_ai_endpoint+ for marker in ["_", "vector_store", "graph", "create_memory", "init"]+ )+ span_kind = SpanKind.INTERNAL if is_internal else SpanKind.CLIENT++ # Create optimized context+ ctx = Mem0Context(+ instance, args, kwargs, version, environment, application_name+ )+ current_context = context.get_current()++ with tracer.start_as_current_span(+ gen_ai_endpoint, kind=span_kind, context=current_context+ ) as span:+ span_context = set_span_in_context(span, context=current_context)+ token = context.attach(span_context)++ start_time = time.perf_counter()++ try:+ # Execute async operation (no threading patches needed for async)+ response = wrapped(*args, **kwargs)++ # Calculate duration+ duration = time.perf_counter() - start_time++ # Set span attributes+ set_mem0_span_attributes(span, gen_ai_endpoint, ctx, response)+ span.set_attribute("gen_ai.client.operation.duration", duration)++ # Set content attributes if enabled+ set_mem0_content_attributes(+ span, gen_ai_endpoint, ctx, response, capture_message_content+ )++ span.set_status(Status(StatusCode.OK))+ return response++ except Exception as e:+ duration = time.perf_counter() - start_time+ span.set_attribute("gen_ai.client.operation.duration", duration)++ try:+ set_mem0_span_attributes(span, gen_ai_endpoint, ctx)+ except Exception as attr_error:+ logger.debug(+ "Failed to set span attributes on error: %s", attr_error+ )++ handle_exception(span, e)+ raise++ finally:+ context.detach(token)++ return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/mem0/mem0.py b/sdk/python/src/openlit/instrumentation/mem0/mem0.pyindex 1641d247..e6593566 100644--- a/sdk/python/src/openlit/instrumentation/mem0/mem0.py+++ b/sdk/python/src/openlit/instrumentation/mem0/mem0.py@@ -1,20 +1,20 @@# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument"""-Module for monitoring mem0 applications.+Optimized sync mem0 instrumentation following OpenLIT Framework Guide."""-import logging-from opentelemetry.trace import SpanKind, Status, StatusCode-from opentelemetry.sdk.resources import (- SERVICE_NAME,- TELEMETRY_SDK_NAME,- DEPLOYMENT_ENVIRONMENT,-)+import time+from opentelemetry.trace import SpanKind, Status, StatusCode, set_span_in_context+from opentelemetry import context+from opentelemetry.instrumentation.utils import _SUPPRESS_INSTRUMENTATION_KEYfrom openlit.__helpers import handle_exception-from openlit.semcov import SemanticConvention--# Initialize logger for logging potential issues and operations-logger = logging.getLogger(__name__)+from openlit.instrumentation.mem0.utils import (+ patch_concurrent_futures_context,+ Mem0Context,+ set_mem0_span_attributes,+ set_mem0_content_attributes,+ logger,+)def mem0_wrap(@@ -27,110 +27,87 @@ def mem0_wrap(capture_message_content,):"""- Creates a wrapper around a function call to trace and log its execution metrics.-- This function wraps any given function to measure its execution time,- log its operation, and trace its execution using OpenTelemetry.-- Parameters:- - gen_ai_endpoint (str): A descriptor or name for the endpoint being traced.- - version (str): The version of the mem0 application.- - environment (str): The deployment environment (e.g., 'production', 'development').- - application_name (str): Name of the mem0 application.- - tracer (opentelemetry.trace.Tracer): The tracer object used for OpenTelemetry tracing.- - pricing_info (dict): Information about the pricing for internal metrics (currently not used).- - capture_message_content (bool): Flag indicating whether to trace the content of the response.+ Optimized wrapper for sync mem0 operations with proper hierarchy and performance.Returns:- - function: A higher-order function that takes a function 'wrapped' and returns- a new function that wraps 'wrapped' with additional tracing and logging.+ Wrapper function for mem0 operations"""def wrapper(wrapped, instance, args, kwargs):"""- An inner wrapper function that executes the wrapped function, measures execution- time, and records trace data using OpenTelemetry.-- Parameters:- - wrapped (Callable): The original function that this wrapper will execute.- - instance (object): The instance to which the wrapped function belongs. This- is used for instance methods. For static and classmethods,- this may be None.- - args (tuple): Positional arguments passed to the wrapped function.- - kwargs (dict): Keyword arguments passed to the wrapped function.-- Returns:- - The result of the wrapped function call.-- The wrapper initiates a span with the provided tracer, sets various attributes- on the span based on the function's execution and response, and ensures- errors are handled and logged appropriately.+ Inner wrapper with comprehensive tracing and optimized performance."""+ # Suppression check+ if context.get_value(_SUPPRESS_INSTRUMENTATION_KEY):+ return wrapped(*args, **kwargs)++ # Handle Memory.__init__ to create parent span for initialization operations+ if gen_ai_endpoint == "memory init":+ with tracer.start_as_current_span("memory init", kind=SpanKind.INTERNAL):+ return wrapped(*args, **kwargs)++ # Determine span type efficiently+ is_internal = any(+ marker in gen_ai_endpoint+ for marker in ["_", "vector_store", "graph", "create_memory", "init"]+ )+ span_kind = SpanKind.INTERNAL if is_internal else SpanKind.CLIENT++ # Create optimized context+ ctx = Mem0Context(+ instance, args, kwargs, version, environment, application_name+ )+ current_context = context.get_current()+with tracer.start_as_current_span(- gen_ai_endpoint, kind=SpanKind.CLIENT+ gen_ai_endpoint, kind=span_kind, context=current_context) as span:- response = wrapped(*args, **kwargs)+ span_context = set_span_in_context(span, context=current_context)+ token = context.attach(span_context)++ start_time = time.perf_counter()try:- span.set_attribute(TELEMETRY_SDK_NAME, "openlit")- span.set_attribute(- SemanticConvention.GEN_AI_SYSTEM,- SemanticConvention.GEN_AI_SYSTEM_MEM0,- )- span.set_attribute(SemanticConvention.GEN_AI_ENDPOINT, gen_ai_endpoint)- span.set_attribute(DEPLOYMENT_ENVIRONMENT, environment)- span.set_attribute(- SemanticConvention.GEN_AI_OPERATION,- SemanticConvention.GEN_AI_OPERATION_TYPE_FRAMEWORK,+ # Apply threading context fix for all CLIENT operations (mem0 operations use ThreadPoolExecutor)+ if span_kind == SpanKind.CLIENT:+ restore_patch = patch_concurrent_futures_context(span_context)+ try:+ response = wrapped(*args, **kwargs)+ finally:+ restore_patch()+ else:+ response = wrapped(*args, **kwargs)++ # Calculate duration+ duration = time.perf_counter() - start_time++ # Set span attributes+ set_mem0_span_attributes(span, gen_ai_endpoint, ctx, response)+ span.set_attribute("gen_ai.client.operation.duration", duration)++ # Set content attributes if enabled+ set_mem0_content_attributes(+ span, gen_ai_endpoint, ctx, response, capture_message_content)- span.set_attribute(SERVICE_NAME, application_name)-- if gen_ai_endpoint == "mem0.memory_add":- span.set_attribute(- SemanticConvention.DB_METADATA, str(kwargs.get("metadata", ""))- )- if response:- span.set_attribute(- SemanticConvention.GEN_AI_DATA_SOURCES, len(response)- )-- elif gen_ai_endpoint == "mem0.memory_get_all":- if response:- span.set_attribute(- SemanticConvention.GEN_AI_DATA_SOURCES, len(response)- )-- elif gen_ai_endpoint == "mem0.memory_get":- if response:- span.set_attribute(- SemanticConvention.GEN_AI_DATA_SOURCES, len(response)- )-- elif gen_ai_endpoint == "mem0.memory_search":- span.set_attribute(- SemanticConvention.DB_STATEMENT, kwargs.get("query", "")- )-- elif gen_ai_endpoint == "mem0.memory_update":- span.set_attribute(- SemanticConvention.DB_UPDATE_ID, kwargs.get("memory_id", "")- )-- elif gen_ai_endpoint == "mem0.memory_delete":- span.set_attribute(- SemanticConvention.DB_DELETE_ID, kwargs.get("memory_id", "")- )span.set_status(Status(StatusCode.OK))-- # Return original responsereturn responseexcept Exception as e:+ duration = time.perf_counter() - start_time+ span.set_attribute("gen_ai.client.operation.duration", duration)++ try:+ set_mem0_span_attributes(span, gen_ai_endpoint, ctx)+ except Exception as attr_error:+ logger.debug(+ "Failed to set span attributes on error: %s", attr_error+ )+handle_exception(span, e)- logger.error("Error in trace creation: %s", e)+ raise- # Return original response- return response+ finally:+ context.detach(token)return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/mem0/utils.py b/sdk/python/src/openlit/instrumentation/mem0/utils.pynew file mode 100644index 00000000..df135b8c--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/mem0/utils.py@@ -0,0 +1,301 @@+# pylint: disable=duplicate-code+"""+Shared utilities for mem0 instrumentation.+"""++import concurrent.futures+import functools+import json+import logging+from typing import Any, Dict, Optional++from opentelemetry import context+from opentelemetry.sdk.resources import (+ SERVICE_NAME,+ TELEMETRY_SDK_NAME,+ DEPLOYMENT_ENVIRONMENT,+)+from openlit.semcov import SemanticConvention++# Initialize logger+logger = logging.getLogger(__name__)+++# Threading context propagation utility+def patch_concurrent_futures_context(current_context):+ """+ Patches ThreadPoolExecutor.submit to propagate OpenTelemetry context+ across thread boundaries for proper mem0 span hierarchy.++ Args:+ current_context: OpenTelemetry context to propagate++ Returns:+ Function to restore original ThreadPoolExecutor.submit+ """+ original_submit = concurrent.futures.ThreadPoolExecutor.submit++ def patched_submit(executor_self, fn, *args, **kwargs):+ @functools.wraps(fn)+ def context_wrapped_fn(*fn_args, **fn_kwargs):+ token = context.attach(current_context)+ try:+ return fn(*fn_args, **fn_kwargs)+ finally:+ context.detach(token)++ return original_submit(executor_self, context_wrapped_fn, *args, **kwargs)++ concurrent.futures.ThreadPoolExecutor.submit = patched_submit+ return lambda: setattr(+ concurrent.futures.ThreadPoolExecutor, "submit", original_submit+ )+++class Mem0Context:+ """Optimized context object with lazy loading and __slots__ for performance."""++ __slots__ = (+ "instance",+ "args",+ "kwargs",+ "version",+ "environment",+ "application_name",+ "_user_id",+ "_agent_id",+ "_run_id",+ "_metadata",+ "_memory_type",+ "_messages",+ )++ def __init__(self, instance, args, kwargs, version, environment, application_name):+ self.instance = instance+ self.args = args+ self.kwargs = kwargs+ self.version = version+ self.environment = environment+ self.application_name = application_name++ # Lazy-loaded cached properties+ self._user_id = None+ self._agent_id = None+ self._run_id = None+ self._metadata = None+ self._memory_type = None+ self._messages = None++ @property+ def user_id(self) -> Optional[str]:+ """Get user_id with lazy loading."""+ if self._user_id is None:+ self._user_id = self.kwargs.get("user_id")+ return self._user_id++ @property+ def agent_id(self) -> Optional[str]:+ """Get agent_id with lazy loading."""+ if self._agent_id is None:+ self._agent_id = self.kwargs.get("agent_id")+ return self._agent_id++ @property+ def run_id(self) -> Optional[str]:+ """Get run_id with lazy loading."""+ if self._run_id is None:+ self._run_id = self.kwargs.get("run_id")+ return self._run_id++ @property+ def metadata(self) -> Optional[Dict[str, Any]]:+ """Get metadata with lazy loading."""+ if self._metadata is None:+ self._metadata = self.kwargs.get("metadata")+ return self._metadata++ @property+ def memory_type(self) -> Optional[str]:+ """Get memory_type with lazy loading."""+ if self._memory_type is None:+ self._memory_type = self.kwargs.get("memory_type")+ return self._memory_type++ @property+ def messages(self) -> Any:+ """Get messages with lazy loading."""+ if self._messages is None:+ self._messages = self.args[0] if self.args else self.kwargs.get("messages")+ return self._messages+++def set_mem0_span_attributes(+ span, operation_name: str, ctx: Mem0Context, response: Any = None+):+ """Set optimized span attributes with batched operations."""++ # Batch core attributes for performance+ core_attributes = {+ TELEMETRY_SDK_NAME: "openlit",+ SemanticConvention.GEN_AI_SYSTEM: SemanticConvention.GEN_AI_SYSTEM_MEM0,+ SemanticConvention.GEN_AI_ENDPOINT: operation_name,+ DEPLOYMENT_ENVIRONMENT: ctx.environment,+ SERVICE_NAME: ctx.application_name,+ SemanticConvention.GEN_AI_SDK_VERSION: ctx.version,+ }++ # Set core attributes in batch+ for key, value in core_attributes.items():+ span.set_attribute(key, value)++ # Determine operation type efficiently+ is_internal = any(+ marker in operation_name+ for marker in ["_", "vector_store", "graph", "create_memory", "procedural"]+ )++ operation_type = (+ SemanticConvention.GEN_AI_OPERATION_TYPE_FRAMEWORK+ if is_internal+ else SemanticConvention.GEN_AI_OPERATION_TYPE_MEMORY+ )+ span.set_attribute(SemanticConvention.GEN_AI_OPERATION, operation_type)++ # Component type mapping for internal operations+ if is_internal:+ component_map = {+ "vector_store": "vector_store",+ "graph": "graph_store",+ "create_memory": "memory_creation",+ "procedural": "procedural_memory",+ }+ for key, component_type in component_map.items():+ if key in operation_name:+ span.set_attribute(+ SemanticConvention.GEN_AI_FRAMEWORK_COMPONENT_TYPE, component_type+ )+ break++ # Session attributes+ session_attrs = {}+ if ctx.user_id:+ session_attrs[SemanticConvention.GEN_AI_USER_ID] = ctx.user_id+ if ctx.agent_id:+ session_attrs[SemanticConvention.GEN_AI_AGENT_ID] = ctx.agent_id+ if ctx.run_id:+ session_attrs[SemanticConvention.GEN_AI_RUN_ID] = ctx.run_id++ for key, value in session_attrs.items():+ span.set_attribute(key, value)++ # Memory-specific attributes+ if ctx.memory_type:+ span.set_attribute(SemanticConvention.GEN_AI_MEMORY_TYPE, ctx.memory_type)++ if ctx.metadata:+ try:+ span.set_attribute(+ SemanticConvention.GEN_AI_MEMORY_METADATA, json.dumps(ctx.metadata)+ )+ except Exception:+ span.set_attribute(+ SemanticConvention.GEN_AI_MEMORY_METADATA, str(ctx.metadata)+ )++ # Operation-specific attributes+ _set_operation_attributes(span, operation_name, ctx)++ # Response attributes+ _set_response_attributes(span, response)+++def _set_operation_attributes(span, operation_name: str, ctx: Mem0Context):+ """Set operation-specific attributes efficiently."""+ if "add" in operation_name:+ if "infer" in ctx.kwargs:+ span.set_attribute(+ SemanticConvention.GEN_AI_MEMORY_INFER, ctx.kwargs["infer"]+ )+ if ctx.messages:+ try:+ count = (+ len(ctx.messages) if isinstance(ctx.messages, (list, tuple)) else 1+ )+ span.set_attribute(SemanticConvention.GEN_AI_MEMORY_COUNT, count)+ except Exception:+ pass++ elif "search" in operation_name:+ search_attrs = {}+ if "query" in ctx.kwargs:+ search_attrs[SemanticConvention.GEN_AI_MEMORY_SEARCH_QUERY] = ctx.kwargs[+ "query"+ ]+ if "limit" in ctx.kwargs:+ search_attrs[SemanticConvention.GEN_AI_MEMORY_SEARCH_LIMIT] = ctx.kwargs[+ "limit"+ ]+ if "threshold" in ctx.kwargs:+ search_attrs[SemanticConvention.GEN_AI_MEMORY_SEARCH_THRESHOLD] = (+ ctx.kwargs["threshold"]+ )++ for key, value in search_attrs.items():+ span.set_attribute(key, value)++ elif "update" in operation_name and "memory_id" in ctx.kwargs:+ span.set_attribute(SemanticConvention.DB_UPDATE_ID, ctx.kwargs["memory_id"])++ elif "delete" in operation_name and "memory_id" in ctx.kwargs:+ span.set_attribute(SemanticConvention.DB_DELETE_ID, ctx.kwargs["memory_id"])++ elif "get" in operation_name and "memory_id" in ctx.kwargs:+ span.set_attribute(SemanticConvention.DB_OPERATION_ID, ctx.kwargs["memory_id"])+++def _set_response_attributes(span, response: Any):+ """Set response attributes for business intelligence."""+ if not response:+ return++ try:+ if isinstance(response, (list, tuple)):+ count = len(response)+ elif isinstance(response, dict):+ count = len(response.get("results", [])) if response.get("results") else 1+ else:+ count = 1++ span.set_attribute(+ SemanticConvention.GEN_AI_MEMORY_OPERATION_RESULT_COUNT, count+ )+ span.set_attribute(SemanticConvention.GEN_AI_DATA_SOURCES, count)+ except Exception:+ pass+++def set_mem0_content_attributes(+ span, operation_name: str, ctx: Mem0Context, response: Any, capture_content: bool+):+ """Set content attributes if enabled."""+ if not capture_content:+ return++ try:+ if ctx.messages and "add" in operation_name:+ content = (+ json.dumps(ctx.messages)+ if isinstance(ctx.messages, (list, dict))+ else str(ctx.messages)+ )+ span.set_attribute(SemanticConvention.GEN_AI_CONTENT_PROMPT, content)++ if response and "search" in operation_name:+ content = (+ json.dumps(response)+ if isinstance(response, (dict, list))+ else str(response)+ )+ span.set_attribute(SemanticConvention.GEN_AI_CONTENT_COMPLETION, content)+ except Exception as e:+ logger.debug("Failed to capture content: %s", e)diff --git a/sdk/python/src/openlit/semcov/__init__.py b/sdk/python/src/openlit/semcov/__init__.pyindex a0506d9b..aca5db79 100644--- a/sdk/python/src/openlit/semcov/__init__.py+++ b/sdk/python/src/openlit/semcov/__init__.py@@ -540,6 +540,25 @@ class SemanticConvention:# Session and conversation trackingGEN_AI_SESSION_ID = "gen_ai.session.id"GEN_AI_USER_ID = "gen_ai.user.id"+ GEN_AI_RUN_ID = "gen_ai.run.id"++ # Memory operation types+ GEN_AI_OPERATION_TYPE_MEMORY = "memory"+ GEN_AI_OPERATION_TYPE_MEMORY_ADD = "memory_add"+ GEN_AI_OPERATION_TYPE_MEMORY_SEARCH = "memory_search"+ GEN_AI_OPERATION_TYPE_MEMORY_GET = "memory_get"+ GEN_AI_OPERATION_TYPE_MEMORY_UPDATE = "memory_update"+ GEN_AI_OPERATION_TYPE_MEMORY_DELETE = "memory_delete"++ # Memory-specific attributes+ GEN_AI_MEMORY_TYPE = "gen_ai.memory.type"+ GEN_AI_MEMORY_METADATA = "gen_ai.memory.metadata"+ GEN_AI_MEMORY_INFER = "gen_ai.memory.infer"+ GEN_AI_MEMORY_COUNT = "gen_ai.memory.count"+ GEN_AI_MEMORY_SEARCH_QUERY = "gen_ai.memory.search.query"+ GEN_AI_MEMORY_SEARCH_LIMIT = "gen_ai.memory.search.limit"+ GEN_AI_MEMORY_SEARCH_THRESHOLD = "gen_ai.memory.search.threshold"+ GEN_AI_MEMORY_OPERATION_RESULT_COUNT = "gen_ai.memory.operation.result_count"# Agent lifecycle phasesGEN_AI_AGENT_LIFECYCLE_PHASE = "gen_ai.agent.lifecycle.phase"diff --git a/sdk/python/tests/test_ai21.py b/sdk/python/tests/test_ai21.pyindex 45e2a2e3..58eb7dcf 100644--- a/sdk/python/tests/test_ai21.py+++ b/sdk/python/tests/test_ai21.py@@ -2,7 +2,7 @@"""This module contains tests for AI21 functionality using the AI21 Python library.-Tests cover various API endpoints, including chat and embeddings.+Tests cover various API endpoints, including chat and embeddings.These tests validate integration with OpenLIT.Environment Variables:@@ -24,12 +24,15 @@ sync_client = AI21Client()async_client = AsyncAI21Client()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-ai21-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-ai21-test"+)MESSAGES = [ChatMessage(content="hi", role="user"),]+def test_sync_ai21_chat():"""Tests synchronous chat.@@ -45,6 +48,7 @@ def test_sync_ai21_chat():)assert isinstance(response.id, str)+def test_sync_ai21_chat_stream():"""Tests synchronous chat streaming.@@ -63,6 +67,7 @@ def test_sync_ai21_chat_stream():assert isinstance(response.id, str)return+def test_sync_ai21_chat_rag():"""Tests synchronous chat rag.@@ -82,6 +87,7 @@ def test_sync_ai21_chat_rag():)assert isinstance(response.id, str)+@pytest.mark.asyncioasync def test_async_ai21_test():"""diff --git a/sdk/python/tests/test_anthropic.py b/sdk/python/tests/test_anthropic.pyindex 1ecbdc8d..b53f5745 100644--- a/sdk/python/tests/test_anthropic.py+++ b/sdk/python/tests/test_anthropic.py@@ -2,7 +2,7 @@"""This module contains tests for Anthropic functionality using the Anthropic Python library.-Tests cover various API endpoints, including chat.+Tests cover various API endpoints, including chat.These tests validate integration with OpenLIT.Environment Variables:@@ -23,7 +23,11 @@ sync_client = Anthropic()async_client = AsyncAnthropic()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-anthropic-test")+openlit.init(+ environment="openlit-python-testing",+ application_name="openlit-python-anthropic-test",+)+def test_sync_anthropic_messages():"""@@ -44,7 +48,7 @@ def test_sync_anthropic_messages():],model="claude-3-haiku-20240307",)- assert message.type == 'message'+ assert message.type == "message"# pylint: disable=broad-exception-caughtexcept Exception as e:@@ -53,6 +57,7 @@ def test_sync_anthropic_messages():else:raise+@pytest.mark.asyncioasync def test_async_anthropic_messages():"""@@ -73,7 +78,7 @@ async def test_async_anthropic_messages():],model="claude-3-haiku-20240307",)- assert message.type == 'message'+ assert message.type == "message"# pylint: disable=broad-exception-caughtexcept Exception as e:diff --git a/sdk/python/tests/test_assemblyai.py b/sdk/python/tests/test_assemblyai.pyindex c4148c7a..efeecf40 100644--- a/sdk/python/tests/test_assemblyai.py+++ b/sdk/python/tests/test_assemblyai.py@@ -2,7 +2,7 @@"""This module contains tests for Assembly AI functionality using the Assembly AI Python library.-Tests cover various API endpoints.+Tests cover various API endpoints.These tests validate integration with OpenLIT.Environment Variables:@@ -18,9 +18,10 @@ import openlit# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-python-testing",- application_name="openlit-python-assemblyai-test"+ application_name="openlit-python-assemblyai-test",)+def test_sync_assemblyai_transcribe():"""Tests synchronous transcribe.@@ -30,9 +31,7 @@ def test_sync_assemblyai_transcribe():"""try:- audio_file = (- "https://audio-samples.github.io/samples/mp3/blizzard_unconditional/sample-0.mp3"- )+ audio_file = "https://audio-samples.github.io/samples/mp3/blizzard_unconditional/sample-0.mp3"transcriber = aai.Transcriber()config = aai.TranscriptionConfig(speech_model=aai.SpeechModel.nano)diff --git a/sdk/python/tests/test_astra.py b/sdk/python/tests/test_astra.pyindex 1b216b9a..0129ed09 100644--- a/sdk/python/tests/test_astra.py+++ b/sdk/python/tests/test_astra.py@@ -3,7 +3,7 @@This module contains tests for AstraDB functionality using the AstraDB Python library.Tests cover various API endpoints, including create_collection, add, query,-upsert, update, get, peek and delete.+upsert, update, get, peek and delete.These tests validate integration with OpenLIT.Note: Ensure the environment is properly configured for AstraDB access and OpenLIT monitoring@@ -16,12 +16,13 @@ import openlit# Initialize the clientclient = DataAPIClient(os.getenv("ASTRA_DB_APPLICATION_TOKEN"))-db = client.get_database_by_api_endpoint(- os.getenv("ASTRA_DB_API_ENDPOINT")-)+db = client.get_database_by_api_endpoint(os.getenv("ASTRA_DB_API_ENDPOINT"))# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-astra-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-astra-test"+)+def test_db_astra():"""@@ -40,31 +41,33 @@ def test_db_astra():# assert collection.name == "openlit"collection = db.get_collection("openlit")- response = collection.insert_one({"summary": "I was flying", "$vector": [-0.4, 0.7, 0]})+ response = collection.insert_one(+ {"summary": "I was flying", "$vector": [-0.4, 0.7, 0]}+ )assert isinstance(response.inserted_id, str)response = collection.insert_many(- [- {- "tags": ["food"],- "summary": "A dinner on the Moon",- "$vector": [0.2, -0.3, -0.5],- },- {- "summary": "Riding the waves",- "tags": ["sport"],- "$vector": [0, 0.2, 1],- },- {- "summary": "Friendly aliens in town",- "tags": ["scifi"],- "$vector": [-0.3, 0, 0.8],- },- {- "summary": "Meeting Beethoven at the dentist",- "$vector": [0.2, 0.6, 0],- },- ],+ [+ {+ "tags": ["food"],+ "summary": "A dinner on the Moon",+ "$vector": [0.2, -0.3, -0.5],+ },+ {+ "summary": "Riding the waves",+ "tags": ["sport"],+ "$vector": [0, 0.2, 1],+ },+ {+ "summary": "Friendly aliens in town",+ "tags": ["scifi"],+ "$vector": [-0.3, 0, 0.8],+ },+ {+ "summary": "Meeting Beethoven at the dentist",+ "$vector": [0.2, 0.6, 0],+ },+ ],)assert isinstance(response.inserted_ids, list)@@ -96,11 +99,11 @@ def test_db_astra():include_similarity=True,)for response in cursor:- assert isinstance(response['summary'], str)+ assert isinstance(response["summary"], str)response = collection.replace_one(- {"tag": {"$exists": True}}, # filter- {"summary": "Pest"}, # replacement+ {"tag": {"$exists": True}}, # filter+ {"summary": "Pest"}, # replacement)assert isinstance(response.update_info["n"], int)diff --git a/sdk/python/tests/test_azure_ai_inference.py b/sdk/python/tests/test_azure_ai_inference.pyindex 7cd80743..4105fabc 100644--- a/sdk/python/tests/test_azure_ai_inference.py+++ b/sdk/python/tests/test_azure_ai_inference.py@@ -37,6 +37,7 @@ sync_embed_client = EmbeddingsClient(# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-testing", application_name="openlit-python-test")+def test_sync_chat_completions():"""Tests synchronous chat completions with the "microsoft/Phi-4-mini-reasoning" model.@@ -53,9 +54,10 @@ def test_sync_chat_completions():model="microsoft/Phi-4-mini-reasoning",temperature=1.0,max_tokens=1000,- top_p=1.0+ top_p=1.0,)- assert response["object"] == 'chat.completion'+ assert response["object"] == "chat.completion"+def test_sync_embeddings():"""@@ -67,11 +69,13 @@ def test_sync_embeddings():response = sync_embed_client.embed(input=["LLM Observability", "Monitor GPUs"],- model="openai/text-embedding-3-small"+ model="openai/text-embedding-3-small",)- assert response.data[0]["object"] == 'embedding'+ assert response.data[0]["object"] == "embedding"+from azure.ai.inference.aio import ChatCompletionsClient, EmbeddingsClient+async_chat_client = ChatCompletionsClient(endpoint="https://models.github.ai/inference",credential=AzureKeyCredential(os.getenv("AZURE_AI_INFERENCE_API_TOKEN")),@@ -82,6 +86,7 @@ async_embed_client = EmbeddingsClient(credential=AzureKeyCredential(os.getenv("AZURE_AI_INFERENCE_API_TOKEN")),)+@pytest.mark.asyncioasync def test_async_chat_completions():"""@@ -99,9 +104,10 @@ async def test_async_chat_completions():model="microsoft/Phi-4-mini-reasoning",temperature=1.0,max_tokens=1000,- top_p=1.0+ top_p=1.0,)- assert response["object"] == 'chat.completion'+ assert response["object"] == "chat.completion"+@pytest.mark.asyncioasync def test_async_embeddings():@@ -114,6 +120,6 @@ async def test_async_embeddings():response = await async_embed_client.embed(input=["LLM Observability", "Monitor GPUs"],- model="openai/text-embedding-3-small"+ model="openai/text-embedding-3-small",)- assert response.data[0]["object"] == 'embedding'+ assert response.data[0]["object"] == "embedding"diff --git a/sdk/python/tests/test_chroma.py b/sdk/python/tests/test_chroma.pyindex 774f8a4c..0328eaf0 100644--- a/sdk/python/tests/test_chroma.py+++ b/sdk/python/tests/test_chroma.py@@ -3,7 +3,7 @@This module contains tests for ChromaDB functionality using the ChromaDB Python library.Tests cover various API endpoints, including create_collection, add, query,-upsert, update, get, peek and delete.+upsert, update, get, peek and delete.These tests validate integration with OpenLIT.Note: Ensure the environment is properly configured for ChromaDB access and OpenLIT monitoring@@ -19,6 +19,7 @@ chroma_client = chromadb.Client()# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-testing", application_name="openlit-python-test")+def test_db_chroma():"""Tests basic operations within a ChromaDB collection.@@ -35,7 +36,7 @@ def test_db_chroma():expected document ids.- A delete operation targets a specific document by id, and successful deletionis implicitly verified by the absence of an error response.-+The test ensures the basic CRUD operations perform as expected in ChromaDB.Raises:AssertionError: If the responses from ChromaDB operations do not meet the expected outcomes.@@ -43,26 +44,20 @@ def test_db_chroma():# Create a new collection named "openlit"collection = chroma_client.create_collection(name="openlit")- assert collection.name == 'openlit'+ assert collection.name == "openlit"# Add documents to the collectiondb_add = collection.add(documents=["This is a document", "This is another document"],metadatas=[{"source": "my_source"}, {"source": "my_source"}],- ids=["id1", "id2"]+ ids=["id1", "id2"],)assert db_add is None# Query the documents in the collection- db_query = collection.query(- query_texts=["This is a query document"],- n_results=2- )- assert db_query["ids"] == [['id1', 'id2']]+ db_query = collection.query(query_texts=["This is a query document"], n_results=2)+ assert db_query["ids"] == [["id1", "id2"]]# Delete a document from the collection- db_delete = collection.delete(- ids=["id2"],- where={"source": "my_source"}- )+ db_delete = collection.delete(ids=["id2"], where={"source": "my_source"})assert db_delete is Nonediff --git a/sdk/python/tests/test_cohere.py b/sdk/python/tests/test_cohere.pyindex 07bb430b..dd4240bc 100644--- a/sdk/python/tests/test_cohere.py+++ b/sdk/python/tests/test_cohere.py@@ -2,7 +2,7 @@"""This module contains tests for Cohere functionality using the Cohere Python library.-Tests cover various API endpoints, including chat and embeddings.+Tests cover various API endpoints, including chat and embeddings.These tests validate integration with OpenLIT.Environment Variables:@@ -19,7 +19,9 @@ import openlitsync_client = cohere.Client()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-cohere-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-cohere-test"+)def test_embed():diff --git a/sdk/python/tests/test_controlflow.py b/sdk/python/tests/test_controlflow.pyindex 174a33cb..46e26830 100644--- a/sdk/python/tests/test_controlflow.py+++ b/sdk/python/tests/test_controlflow.py@@ -2,7 +2,7 @@"""This module contains tests for ControlFlow functionality using the ControlFlow Python library.-Tests cover various API endpoints, including chat and embeddings.+Tests cover various API endpoints, including chat and embeddings.These tests validate integration with OpenLIT.Environment Variables:@@ -18,9 +18,10 @@ import openlit# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-python-testing",- application_name="openlit-python-controlflow-test"+ application_name="openlit-python-controlflow-test",)+def test_sync_controlflow_agent():"""Tests Agent Creation.@@ -36,6 +37,7 @@ def test_sync_controlflow_agent():)assert isinstance(agent, cf.Agent)+def test_sync_controlflow_task():"""Tests Task Creation.diff --git a/sdk/python/tests/test_elevenlabs.py b/sdk/python/tests/test_elevenlabs.pyindex b83d3e2c..55258003 100644--- a/sdk/python/tests/test_elevenlabs.py+++ b/sdk/python/tests/test_elevenlabs.py@@ -2,7 +2,7 @@"""This module contains tests for ElevenLabs functionality using the ElevenLabs Python library.-Tests cover various API endpoints.+Tests cover various API endpoints.These tests validate integration with OpenLIT.Environment Variables:@@ -25,7 +25,7 @@ async_client = AsyncElevenLabs()# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-python-testing",- application_name="openlit-python-elevenlabs-test"+ application_name="openlit-python-elevenlabs-test",)@@ -53,6 +53,7 @@ def test_sync_elevenlabs_t2s():else:raise+@pytest.mark.asyncioasync def test_async_elevenlabs_t2s():"""diff --git a/sdk/python/tests/test_embedchain.py b/sdk/python/tests/test_embedchain.pyindex e6e8a98a..6e432d3c 100644--- a/sdk/python/tests/test_embedchain.py+++ b/sdk/python/tests/test_embedchain.py@@ -19,43 +19,40 @@ import openlit# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-python-testing",- application_name="openlit-python-embedchain-test"+ application_name="openlit-python-embedchain-test",)config = {- 'app': {- 'config': {- 'name': 'openlit-testing'- }+ "app": {"config": {"name": "openlit-testing"}},+ "llm": {+ "provider": "openai",+ "config": {+ "model": "gpt-3.5-turbo",+ "temperature": 0.5,+ "max_tokens": 1,+ "top_p": 1,+ "stream": False,+ "api_key": os.getenv("OPENAI_API_TOKEN"),+ },},- 'llm': {- 'provider': 'openai',- 'config': {- 'model': 'gpt-3.5-turbo',- 'temperature': 0.5,- 'max_tokens': 1,- 'top_p': 1,- 'stream': False,- 'api_key': os.getenv("OPENAI_API_TOKEN")- }+ "vectordb": {+ "provider": "chroma",+ "config": {+ "collection_name": "full-stack-app",+ "dir": "db",+ "allow_reset": True,+ },},- 'vectordb': {- 'provider': 'chroma',- 'config': {- 'collection_name': 'full-stack-app',- 'dir': 'db',- 'allow_reset': True- }+ "embedder": {+ "provider": "openai",+ "config": {+ "model": "text-embedding-ada-002",+ "api_key": os.getenv("OPENAI_API_TOKEN"),+ },},- 'embedder': {- 'provider': 'openai',- 'config': {- 'model': 'text-embedding-ada-002',- 'api_key': os.getenv("OPENAI_API_TOKEN")- }- }}+def test_embedchain():"""Verifies the functionality of Embedchain operations including document loading, text splitting,@@ -76,6 +73,7 @@ def test_embedchain():data_sources = app.get_data_sources()assert isinstance(len(data_sources), int)- evals = app.evaluate(["What is the net worth of Elon Musk?",- "How many companies Elon Musk owns?"])+ evals = app.evaluate(+ ["What is the net worth of Elon Musk?", "How many companies Elon Musk owns?"]+ )assert isinstance(evals, dict)diff --git a/sdk/python/tests/test_firecrawl.py b/sdk/python/tests/test_firecrawl.pyindex 218f9ca3..732b0a16 100644--- a/sdk/python/tests/test_firecrawl.py+++ b/sdk/python/tests/test_firecrawl.py@@ -2,7 +2,7 @@"""This module contains tests for FireCrawl functionality using the FireCrawl Python library.-Tests cover various API endpoints, including chat and embeddings.+Tests cover various API endpoints, including chat and embeddings.These tests validate integration with OpenLIT.Environment Variables:@@ -20,7 +20,11 @@ import openlitsync_client = FirecrawlApp(api_key=os.getenv("FIRECRAWL_API_KEY"))# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-firecrawl-test")+openlit.init(+ environment="openlit-python-testing",+ application_name="openlit-python-firecrawl-test",+)+def test_sync_scarpe_url():"""@@ -31,8 +35,8 @@ def test_sync_scarpe_url():"""response = sync_client.scrape_url(- 'https://openlit.io',- formats=['markdown', 'html'],+ "https://openlit.io",+ formats=["markdown", "html"],)assert response.success is Truediff --git a/sdk/python/tests/test_google_ai_studio.py b/sdk/python/tests/test_google_ai_studio.pyindex 5b5360f7..5f674ebf 100644--- a/sdk/python/tests/test_google_ai_studio.py+++ b/sdk/python/tests/test_google_ai_studio.py@@ -3,7 +3,7 @@This module contains tests for Google AI Studio functionality using the google-generativeaiPython library.-Tests cover various API endpoints, including chat.+Tests cover various API endpoints, including chat.These tests validate integration with OpenLIT.Environment Variables:@@ -20,9 +20,7 @@ from google.genai import typesimport openlit# Initialize Google AI Studio client-client = genai.Client(- api_key=os.getenv("GOOGLE_AI_STUDIO_API_TOKEN")-)+client = genai.Client(api_key=os.getenv("GOOGLE_AI_STUDIO_API_TOKEN"))contents = [types.Content(role="user",@@ -36,7 +34,10 @@ generate_content_config = types.GenerateContentConfig()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-testing", application_name="openlit-python-gemini-test")+openlit.init(+ environment="openlit-testing", application_name="openlit-python-gemini-test"+)+def test_sync_generate_content():"""@@ -61,6 +62,7 @@ def test_sync_generate_content():else:raise+@pytest.mark.asyncioasync def test_async_generate_content():"""diff --git a/sdk/python/tests/test_groq.py b/sdk/python/tests/test_groq.pyindex ed6bb84d..3fd8c389 100644--- a/sdk/python/tests/test_groq.py+++ b/sdk/python/tests/test_groq.py@@ -2,7 +2,7 @@"""This module contains tests for Groq functionality using the Groq Python library.-Tests cover various API endpoints, including chat.+Tests cover various API endpoints, including chat.These tests validate integration with OpenLIT.Environment Variables:@@ -18,18 +18,15 @@ from groq import Groq, AsyncGroqimport openlit# Initialize synchronous Groq client-sync_client = Groq(- api_key=os.getenv("GROQ_API_TOKEN")-)+sync_client = Groq(api_key=os.getenv("GROQ_API_TOKEN"))# Initialize asynchronous Groq client-async_client = AsyncGroq(- api_key=os.getenv("GROQ_API_TOKEN")-)+async_client = AsyncGroq(api_key=os.getenv("GROQ_API_TOKEN"))# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-testing", application_name="openlit-python-test")+def test_sync_groq_chat():"""Tests synchronous Chat Completions with the 'llama3-8b-8192' model.@@ -48,9 +45,9 @@ def test_sync_groq_chat():],model="llama3-8b-8192",max_tokens=1,- stream = False,+ stream=False,)- assert chat_completions_resp.object == 'chat.completion'+ assert chat_completions_resp.object == "chat.completion"# pylint: disable=broad-exception-caughtexcept Exception as e:@@ -59,6 +56,7 @@ def test_sync_groq_chat():else:raise+@pytest.mark.asyncioasync def test_async_groq_chat():"""@@ -80,7 +78,7 @@ async def test_async_groq_chat():max_tokens=1,stream=False,)- assert chat_completions_resp.object == 'chat.completion'+ assert chat_completions_resp.object == "chat.completion"# pylint: disable=broad-exception-caughtexcept Exception as e:diff --git a/sdk/python/tests/test_mistral.py b/sdk/python/tests/test_mistral.pyindex 0904fe07..35e9d702 100644--- a/sdk/python/tests/test_mistral.py+++ b/sdk/python/tests/test_mistral.py@@ -2,7 +2,7 @@"""This module contains tests for Mistral functionality using the Mistral Python library.-Tests cover various API endpoints, including chat and embeddings.+Tests cover various API endpoints, including chat and embeddings.These tests validate integration with OpenLIT.Environment Variables:@@ -23,7 +23,10 @@ client = Mistral()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-mistral-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-mistral-test"+)+def test_sync_mistral_chat():"""@@ -45,7 +48,8 @@ def test_sync_mistral_chat():messages=messages,max_tokens=1,)- assert message.object == 'chat.completion'+ assert message.object == "chat.completion"+def test_sync_mistral_embeddings():"""@@ -56,10 +60,11 @@ def test_sync_mistral_embeddings():"""response = client.embeddings.create(- model="mistral-embed",- inputs=["Embed this sentence.", "OpenTelemetry LLM Observability"],+ model="mistral-embed",+ inputs=["Embed this sentence.", "OpenTelemetry LLM Observability"],)- assert response.object == 'list'+ assert response.object == "list"+@pytest.mark.asyncioasync def test_async_mistral():@@ -83,11 +88,11 @@ async def test_async_mistral():messages=messages,max_tokens=1,)- assert message.object == 'chat.completion'+ assert message.object == "chat.completion"# Tests asynchronous embedding creation with the 'mistral-embed' model.response = await client.embeddings.create_async(- model="mistral-embed",- inputs=["Embed this sentence.", "Monitor LLM Applications"],+ model="mistral-embed",+ inputs=["Embed this sentence.", "Monitor LLM Applications"],)- assert response.object == 'list'+ assert response.object == "list"diff --git a/sdk/python/tests/test_openai.py b/sdk/python/tests/test_openai.pyindex bea1cfba..fbaf5bb7 100644--- a/sdk/python/tests/test_openai.py+++ b/sdk/python/tests/test_openai.py@@ -25,7 +25,10 @@ sync_client = OpenAI()async_client = AsyncOpenAI()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-openai-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-openai-test"+)+def test_sync_openai_chat_completions():"""@@ -38,9 +41,10 @@ def test_sync_openai_chat_completions():chat_completions_resp = sync_client.chat.completions.create(model="gpt-3.5-turbo",max_tokens=1,- messages=[{"role": "user", "content": "What is LLM Observability?"}]+ messages=[{"role": "user", "content": "What is LLM Observability?"}],)- assert chat_completions_resp.object == 'chat.completion'+ assert chat_completions_resp.object == "chat.completion"+def test_sync_openai_chat_completions_parse():"""@@ -52,19 +56,26 @@ def test_sync_openai_chat_completions_parse():class User(BaseModel):"""A model to represent a user's details."""+name: strage: int-chat_completions_resp = sync_client.beta.chat.completions.parse(model="gpt-4o-mini",messages=[- {"role": "system", "content": "Extract the user's name and age from the following text."},- {"role": "user", "content": "The user's name is John Doe and he is 30 years old."},+ {+ "role": "system",+ "content": "Extract the user's name and age from the following text.",+ },+ {+ "role": "user",+ "content": "The user's name is John Doe and he is 30 years old.",+ },],response_format=User, # Pass the Pydantic model as the response format)- assert chat_completions_resp.object == 'chat.completion'+ assert chat_completions_resp.object == "chat.completion"+def test_sync_openai_embeddings():"""@@ -77,9 +88,10 @@ def test_sync_openai_embeddings():embeddings_resp = sync_client.embeddings.create(model="text-embedding-ada-002",input="The quick brown fox jumped over the lazy dog",- encoding_format="float"+ encoding_format="float",)- assert embeddings_resp.data[0].object == 'embedding'+ assert embeddings_resp.data[0].object == "embedding"+# def test_sync_openai_fine_tuning_job_creation():# """@@ -102,6 +114,7 @@ def test_sync_openai_embeddings():# rate_limit_code = error_json['error']['code']# print(rate_limit_code)+def test_sync_openai_image_generation():"""Tests synchronous image generation with the 'dall-e-2' model.@@ -111,13 +124,11 @@ def test_sync_openai_image_generation():"""image_generation_resp = sync_client.images.generate(- model='dall-e-2',- prompt='Generate an image of a cat.',- size='256x256',- n=1+ model="dall-e-2", prompt="Generate an image of a cat.", size="256x256", n=1)assert image_generation_resp.created is not None+def test_sync_openai_image_variations():"""Tests synchronous image variation creation with the 'dall-e-2' model@@ -128,12 +139,13 @@ def test_sync_openai_image_variations():image_variation_resp = sync_client.images.create_variation(image=open("tests/test-image-for-openai.png", "rb"),- model='dall-e-2',+ model="dall-e-2",n=1,- size="256x256"+ size="256x256",)assert image_variation_resp.created is not None+def test_sync_openai_audio_speech_create():"""Tests synchronous audio speech generation with the 'tts-1' model.@@ -143,11 +155,11 @@ def test_sync_openai_audio_speech_create():"""audio_speech_resp = sync_client.audio.speech.create(- model='tts-1',- voice='alloy',- input='LLM Observability!')+ model="tts-1", voice="alloy", input="LLM Observability!"+ )assert audio_speech_resp is not None and isinstance(audio_speech_resp, object)+@pytest.mark.asyncioasync def test_async_openai_chat_completions():"""@@ -160,9 +172,10 @@ async def test_async_openai_chat_completions():chat_completions_resp = await async_client.chat.completions.create(model="gpt-3.5-turbo",max_tokens=1,- messages=[{"role": "user", "content": "What is LLM Observability?"}]+ messages=[{"role": "user", "content": "What is LLM Observability?"}],)- assert chat_completions_resp.object == 'chat.completion'+ assert chat_completions_resp.object == "chat.completion"+@pytest.mark.asyncioasync def test_async_openai_chat_completions_parse():@@ -175,18 +188,26 @@ async def test_async_openai_chat_completions_parse():class User(BaseModel):"""A model to represent a user's details."""+name: strage: intchat_completions_resp = await async_client.beta.chat.completions.parse(model="gpt-4o-mini",messages=[- {"role": "system", "content": "Extract the user's name and age from the following text."},- {"role": "user", "content": "The user's name is John Doe and he is 30 years old."},+ {+ "role": "system",+ "content": "Extract the user's name and age from the following text.",+ },+ {+ "role": "user",+ "content": "The user's name is John Doe and he is 30 years old.",+ },],response_format=User, # Pass the Pydantic model as the response format)- assert chat_completions_resp.object == 'chat.completion'+ assert chat_completions_resp.object == "chat.completion"+# @pytest.mark.asyncio# async def test_async_openai_embeddings():diff --git a/sdk/python/tests/test_qdrant.py b/sdk/python/tests/test_qdrant.pyindex a38e6e6b..2393493b 100644--- a/sdk/python/tests/test_qdrant.py+++ b/sdk/python/tests/test_qdrant.py@@ -3,7 +3,7 @@This module contains tests for Qdrant functionality using the Qdrant Python library.Tests cover various API endpoints, including create_collection, add, query,-upsert, update, get and delete.+upsert, update, get and delete.These tests validate integration with OpenLIT.Environment Variables:@@ -26,9 +26,12 @@ client = QdrantClient()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-qdrant-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-qdrant-test"+)COLLECTION_NAME = "openlit"+def test_db_qdrant():"""Tests basic operations within a Qdrant collection.@@ -45,7 +48,7 @@ def test_db_qdrant():expected document ids.- A delete operation targets a specific document by id, and successful deletionis implicitly verified by the absence of an error response.-+The test ensures the basic CRUD operations perform as expected in ChromaDB.Raises:AssertionError: If the responses from ChromaDB operations do not meet the expected outcomes.@@ -61,165 +64,179 @@ def test_db_qdrant():# Upsert to the collectionupsert = client.upsert(- collection_name=COLLECTION_NAME,- wait=True,- points=[- PointStruct(id=1, vector=[0.05, 0.61, 0.76, 0.74], payload={"city": "Berlin"}),- PointStruct(id=2, vector=[0.19, 0.81, 0.75, 0.11], payload={"city": "London"}),- PointStruct(id=3, vector=[0.36, 0.55, 0.47, 0.94], payload={"city": "Moscow"}),- PointStruct(id=4, vector=[0.18, 0.01, 0.85, 0.80], payload={"city": "New York"}),- PointStruct(id=5, vector=[0.24, 0.18, 0.22, 0.44], payload={"city": "Beijing"}),- PointStruct(id=6, vector=[0.35, 0.08, 0.11, 0.44], payload={"city": "Mumbai"}),- ],+ collection_name=COLLECTION_NAME,+ wait=True,+ points=[+ PointStruct(+ id=1, vector=[0.05, 0.61, 0.76, 0.74], payload={"city": "Berlin"}+ ),+ PointStruct(+ id=2, vector=[0.19, 0.81, 0.75, 0.11], payload={"city": "London"}+ ),+ PointStruct(+ id=3, vector=[0.36, 0.55, 0.47, 0.94], payload={"city": "Moscow"}+ ),+ PointStruct(+ id=4, vector=[0.18, 0.01, 0.85, 0.80], payload={"city": "New York"}+ ),+ PointStruct(+ id=5, vector=[0.24, 0.18, 0.22, 0.44], payload={"city": "Beijing"}+ ),+ PointStruct(+ id=6, vector=[0.35, 0.08, 0.11, 0.44], payload={"city": "Mumbai"}+ ),+ ],)- assert upsert.status == 'completed'+ assert upsert.status == "completed"# Set Payload to the collectionset_payload = client.set_payload(- collection_name=COLLECTION_NAME,- payload={- "city": "Vienna",- },- points=[1],+ collection_name=COLLECTION_NAME,+ payload={+ "city": "Vienna",+ },+ points=[1],)- assert set_payload.status == 'completed'+ assert set_payload.status == "completed"# Overwrite Payload to the collectionoverwrite_payload = client.overwrite_payload(- collection_name=COLLECTION_NAME,- payload={- "city": "Toronto",- },- points=[1],+ collection_name=COLLECTION_NAME,+ payload={+ "city": "Toronto",+ },+ points=[1],)- assert overwrite_payload.status == 'completed'+ assert overwrite_payload.status == "completed"# Clear Payload to the collectionclear_payload = client.clear_payload(collection_name=COLLECTION_NAME,points_selector=[1],)- assert clear_payload.status == 'completed'+ assert clear_payload.status == "completed"# Delete Payload to the collectiondelete_payload = client.delete_payload(- collection_name=COLLECTION_NAME,- keys=["city"],- points=[2],+ collection_name=COLLECTION_NAME,+ keys=["city"],+ points=[2],)- assert delete_payload.status == 'completed'+ assert delete_payload.status == "completed"# Upload Points to the collectionupload_points = client.upload_points(- collection_name=COLLECTION_NAME,- points=[- models.PointStruct(- id=1,- payload={- "city": "Toronto",- },- vector=[0.9, 0.1, 0.1, 0.2],- )- ],- max_retries=3,+ collection_name=COLLECTION_NAME,+ points=[+ models.PointStruct(+ id=1,+ payload={+ "city": "Toronto",+ },+ vector=[0.9, 0.1, 0.1, 0.2],+ )+ ],+ max_retries=3,)assert upload_points is None# Update Vectors in the collectionupdate_vectors = client.update_vectors(- collection_name=COLLECTION_NAME,- points=[- models.PointVectors(- id=1,- vector=[0.9, 0.1, 0.1, 0.2],- )- ],+ collection_name=COLLECTION_NAME,+ points=[+ models.PointVectors(+ id=1,+ vector=[0.9, 0.1, 0.1, 0.2],+ )+ ],)- assert update_vectors.status == 'completed'+ assert update_vectors.status == "completed"# Delete Points in the collectiondelt = client.delete(- collection_name=COLLECTION_NAME,- points_selector=models.PointIdsList(- points=[2],- ),+ collection_name=COLLECTION_NAME,+ points_selector=models.PointIdsList(+ points=[2],+ ),)- assert delt.status == 'completed'+ assert delt.status == "completed"# Retrieve vectors from the collectionretrieve = client.retrieve(- collection_name=COLLECTION_NAME,- ids=[4],+ collection_name=COLLECTION_NAME,+ ids=[4],)assert isinstance(retrieve, list)client.create_payload_index(- collection_name=COLLECTION_NAME,- field_name="city",- field_schema="keyword",+ collection_name=COLLECTION_NAME,+ field_name="city",+ field_schema="keyword",)# Scroll vectors from the collectionscroll = client.scroll(- collection_name=COLLECTION_NAME,- scroll_filter=models.Filter(- must=[- models.FieldCondition(key="city", match=models.MatchValue(value="Toronto")),- ]- ),- limit=1,- with_payload=True,- with_vectors=False,+ collection_name=COLLECTION_NAME,+ scroll_filter=models.Filter(+ must=[+ models.FieldCondition(+ key="city", match=models.MatchValue(value="Toronto")+ ),+ ]+ ),+ limit=1,+ with_payload=True,+ with_vectors=False,)assert isinstance(scroll, tuple)# Search vectors from the collectionsearch = client.search(- collection_name=COLLECTION_NAME,- query_filter=models.Filter(- must=[- models.FieldCondition(- key="city",- match=models.MatchValue(- value="London",- ),- )- ]- ),- search_params=models.SearchParams(hnsw_ef=128, exact=False),- query_vector=[0.2, 0.1, 0.9, 0.7],- limit=3,+ collection_name=COLLECTION_NAME,+ query_filter=models.Filter(+ must=[+ models.FieldCondition(+ key="city",+ match=models.MatchValue(+ value="London",+ ),+ )+ ]+ ),+ search_params=models.SearchParams(hnsw_ef=128, exact=False),+ query_vector=[0.2, 0.1, 0.9, 0.7],+ limit=3,)assert isinstance(search, list)# Search groups from the collectionsearch_groups = client.search_groups(- collection_name=COLLECTION_NAME,- query_vector=[1.1, 1.2, 1.3, 1.4],- group_by="city",- limit=4,- group_size=2,+ collection_name=COLLECTION_NAME,+ query_vector=[1.1, 1.2, 1.3, 1.4],+ group_by="city",+ limit=4,+ group_size=2,)assert isinstance(search_groups.groups, list)# Get Recommened vectors from the collectionrecommend = client.recommend(- collection_name=COLLECTION_NAME,- positive=[1, 3],- negative=[4, [0.2, 0.3, 0.4, 0.5]],- strategy=models.RecommendStrategy.AVERAGE_VECTOR,- query_filter=models.Filter(- must=[- models.FieldCondition(- key="city",- match=models.MatchValue(- value="London",- ),- )- ]- ),- limit=3,+ collection_name=COLLECTION_NAME,+ positive=[1, 3],+ negative=[4, [0.2, 0.3, 0.4, 0.5]],+ strategy=models.RecommendStrategy.AVERAGE_VECTOR,+ query_filter=models.Filter(+ must=[+ models.FieldCondition(+ key="city",+ match=models.MatchValue(+ value="London",+ ),+ )+ ]+ ),+ limit=3,)assert isinstance(recommend, list)diff --git a/sdk/python/tests/test_reka.py b/sdk/python/tests/test_reka.pyindex 729ecf8d..f6f8e137 100644--- a/sdk/python/tests/test_reka.py+++ b/sdk/python/tests/test_reka.py@@ -2,7 +2,7 @@"""This module contains tests for Reka AI functionality using the Reka Python library.-Tests cover various API endpoints, including chat.+Tests cover various API endpoints, including chat.These tests validate integration with OpenLIT.Environment Variables:@@ -24,7 +24,10 @@ sync_client = Reka()async_client = AsyncReka()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-reka-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-reka-test"+)+def test_sync_reka_chat():"""@@ -45,7 +48,7 @@ def test_sync_reka_chat():model="reka-core",max_tokens=1,)- assert response.model == 'reka-core'+ assert response.model == "reka-core"# pylint: disable=broad-exception-caughtexcept Exception as e:@@ -54,6 +57,7 @@ def test_sync_reka_chat():else:raise+@pytest.mark.asyncioasync def test_async_reka_chat():"""@@ -74,7 +78,7 @@ async def test_async_reka_chat():model="reka-core",max_tokens=1,)- assert response.model == 'reka-core'+ assert response.model == "reka-core"# pylint: disable=broad-exception-caughtexcept Exception as e:diff --git a/sdk/python/tests/test_together.py b/sdk/python/tests/test_together.pyindex 57104682..4096f8ce 100644--- a/sdk/python/tests/test_together.py+++ b/sdk/python/tests/test_together.py@@ -1,7 +1,7 @@"""This module contains tests for Together functionality using the Together Python library.-Tests cover various API endpoints, including chat and embeddings.+Tests cover various API endpoints, including chat and embeddings.These tests validate integration with OpenLIT.Environment Variables:@@ -22,7 +22,11 @@ sync_client = Together()async_client = AsyncTogether()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-together-test")+openlit.init(+ environment="openlit-python-testing",+ application_name="openlit-python-together-test",+)+def test_sync_together_chat():"""@@ -33,18 +37,15 @@ def test_sync_together_chat():"""try:- response = sync_client.chat.completions.create(+ response = sync_client.chat.completions.create(model="meta-llama/Llama-3.3-70B-Instruct-Turbo",messages=[- {- "role": "user",- "content": "Hi"- },+ {"role": "user", "content": "Hi"},],max_tokens=1,stream=False,)- assert response.model == 'meta-llama/Llama-3.3-70B-Instruct-Turbo'+ assert response.model == "meta-llama/Llama-3.3-70B-Instruct-Turbo"except Exception as e:if "credit_limit" in str(e).lower():@@ -54,6 +55,7 @@ def test_sync_together_chat():else:raise+def test_sync_together_image():"""Tests synchronous image generate.@@ -70,7 +72,7 @@ def test_sync_together_image():height=768,n=1,)- assert response.model == 'black-forest-labs/FLUX.1-dev'+ assert response.model == "black-forest-labs/FLUX.1-dev"except Exception as e:if "credit_limit" in str(e).lower():@@ -80,6 +82,7 @@ def test_sync_together_image():else:raise+@pytest.mark.asyncioasync def test_async_together_chat():"""@@ -90,18 +93,15 @@ async def test_async_together_chat():"""try:- response = await async_client.chat.completions.create(+ response = await async_client.chat.completions.create(model="meta-llama/Llama-3.3-70B-Instruct-Turbo",messages=[- {- "role": "user",- "content": "Hi"- },+ {"role": "user", "content": "Hi"},],max_tokens=1,stream=False,)- assert response.model == 'meta-llama/Llama-3.3-70B-Instruct-Turbo'+ assert response.model == "meta-llama/Llama-3.3-70B-Instruct-Turbo"except Exception as e:if "credit_limit" in str(e).lower():@@ -111,6 +111,7 @@ async def test_async_together_chat():else:raise+@pytest.mark.asyncioasync def test_async_together_image():"""@@ -128,7 +129,7 @@ async def test_async_together_image():height=768,n=1,)- assert response.model == 'black-forest-labs/FLUX.1-dev'+ assert response.model == "black-forest-labs/FLUX.1-dev"except Exception as e:if "credit_limit" in str(e).lower():diff --git a/sdk/python/tests/test_transformers.py b/sdk/python/tests/test_transformers.pyindex feeb9bd6..0a7fe863 100644--- a/sdk/python/tests/test_transformers.py+++ b/sdk/python/tests/test_transformers.py@@ -2,7 +2,7 @@"""This module contains tests for Text Generation functionality in HuggingFace Transformers library.-Tests cover the usage of the Transformers' pipeline for generating text.+Tests cover the usage of the Transformers' pipeline for generating text.These tests validate integration with OpenLIT.Note: Ensure the environment is properly configured for Transformers and OpenLIT monitoring@@ -15,18 +15,22 @@ import openlit# Initialize environment and application name for OpenLIT monitoringopenlit.init(environment="openlit-testing", application_name="openlit-python-test")+def test_text_transformers():"""Test text generation capabilities from HuggingFace Transformers library."""- pipe= pipeline(task="text-generation", model="Qwen/Qwen2.5-1.5B")+ pipe = pipeline(task="text-generation", model="Qwen/Qwen2.5-1.5B")response = pipe("LLM Observability")assert isinstance(response[0]["generated_text"], str)chat = [- {"role": "system", "content": "You are an OpenTelemetry AI Observability expert"},- {"role": "user", "content": "What is Agent Observability?"}+ {+ "role": "system",+ "content": "You are an OpenTelemetry AI Observability expert",+ },+ {"role": "user", "content": "What is Agent Observability?"},]response = pipe(chat, max_new_tokens=100)diff --git a/sdk/python/tests/test_xai.py b/sdk/python/tests/test_xai.pyindex 5b7eea86..d87b8d1e 100644--- a/sdk/python/tests/test_xai.py+++ b/sdk/python/tests/test_xai.py@@ -31,7 +31,10 @@ async_client = AsyncOpenAI()# Initialize environment and application name for OpenLIT monitoring-openlit.init(environment="openlit-python-testing", application_name="openlit-python-xai-test")+openlit.init(+ environment="openlit-python-testing", application_name="openlit-python-xai-test"+)+def test_sync_xai_chat_completions():"""@@ -49,7 +52,7 @@ def test_sync_xai_chat_completions():],max_tokens=1,)- assert response.object == 'chat.completion'+ assert response.object == "chat.completion"# pylint: disable=broad-exception-caughtexcept Exception as e:@@ -58,6 +61,7 @@ def test_sync_xai_chat_completions():else:raise+@pytest.mark.asyncioasync def test_async_xai_chat_completions():"""@@ -75,7 +79,7 @@ async def test_async_xai_chat_completions():],max_tokens=1,)- assert response.object == 'chat.completion'+ assert response.object == "chat.completion"# pylint: disable=broad-exception-caughtexcept Exception as e:diff --git a/src/client/package-lock.json b/src/client/package-lock.jsonindex fc6e519b..7765f50f 100644--- a/src/client/package-lock.json+++ b/src/client/package-lock.json@@ -1,12 +1,12 @@{"name": "openlit",- "version": "1.14.1",+ "version": "1.14.2","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "openlit",- "version": "1.14.1",+ "version": "1.14.2","dependencies": {"@anthropic-ai/sdk": "^0.21.1","@clickhouse/client": "^1.1.0",diff --git a/src/client/package.json b/src/client/package.jsonindex 12582d0d..e48ccbcc 100644--- a/src/client/package.json+++ b/src/client/package.json@@ -1,6 +1,6 @@{"name": "openlit",- "version": "1.14.1",+ "version": "1.14.2","private": true,"scripts": {"dev": "next dev",diff --git a/src/client/scripts/evaluation/evaluate.py b/src/client/scripts/evaluation/evaluate.pyindex ba146a80..293b0170 100644--- a/src/client/scripts/evaluation/evaluate.py+++ b/src/client/scripts/evaluation/evaluate.py@@ -100,8 +100,10 @@ import litellmimport jsonimport sys-def get_system_prompt(threshold_score: float,- prompt: str, contexts: str, response: str) -> str:++def get_system_prompt(+ threshold_score: float, prompt: str, contexts: str, response: str+) -> str:return f"""Output Requirement: List of JSON Objects in JSON array@@ -192,7 +194,7 @@ def llm_response(prompt: str, model: str, api_key=str):{"role": "user", "content": prompt},],temperature=0.0,- response_format={ "type": "json_object" }+ response_format={"type": "json_object"},)try:@@ -201,17 +203,23 @@ def llm_response(prompt: str, model: str, api_key=str):parsed_content = json.loads(content)return parsed_contentexcept Exception as e:- return {- "success": False,- "error": str(e)- }--def measure(api_key: [str], model: Optional[str] = "openai/gpt-4o",- prompt: Optional[str] = None,- contexts: Optional[List[str]] = None,- response: Optional[str] = None, threshold_score: Optional[float] = 0.5):-- llm_prompt = get_system_prompt(threshold_score=threshold_score, prompt=prompt, contexts=contexts, response=response)+ return {"success": False, "error": str(e)}+++def measure(+ api_key: [str],+ model: Optional[str] = "openai/gpt-4o",+ prompt: Optional[str] = None,+ contexts: Optional[List[str]] = None,+ response: Optional[str] = None,+ threshold_score: Optional[float] = 0.5,+):+ llm_prompt = get_system_prompt(+ threshold_score=threshold_score,+ prompt=prompt,+ contexts=contexts,+ response=response,+ )response = llm_response(llm_prompt, model, api_key)return response@@ -221,31 +229,26 @@ def main():try:# Read JSON input from stdininput_json = json.loads(sys.argv[1])-+# Call measure function with parameters from JSONresult = measure(- api_key=input_json.get('api_key'),- model=input_json.get('model'),- prompt=input_json.get('prompt'),- contexts=input_json.get('contexts'),- response=input_json.get('response'),- threshold_score=input_json.get('threshold_score', 0.5)+ api_key=input_json.get("api_key"),+ model=input_json.get("model"),+ prompt=input_json.get("prompt"),+ contexts=input_json.get("contexts"),+ response=input_json.get("response"),+ threshold_score=input_json.get("threshold_score", 0.5),)-+# print(input_json)# Print result as JSON stringprint(json.dumps(result))except json.JSONDecodeError:- print(json.dumps({- "success": False,- "error": "Invalid JSON input"- }))+ print(json.dumps({"success": False, "error": "Invalid JSON input"}))except Exception as e:- print(json.dumps({- "success": False,- "error": str(e)- }))+ print(json.dumps({"success": False, "error": str(e)}))+if __name__ == "__main__":main()diff --git a/temp_analysis/logfire b/temp_analysis/logfiredeleted file mode 160000index da7c0802..00000000--- a/temp_analysis/logfire+++ /dev/null@@ -1 +0,0 @@-Subproject commit da7c0802ac6dbe93a315c0accb0b0efe8d84d17fdiff --git a/temp_analysis/openinference b/temp_analysis/openinferencedeleted file mode 160000index 0864c13f..00000000--- a/temp_analysis/openinference+++ /dev/null@@ -1 +0,0 @@-Subproject commit 0864c13fdfa9e289468ac0a79a2860a155be46dediff --git a/temp_analysis/openllmetry b/temp_analysis/openllmetrydeleted file mode 160000index a7509447..00000000--- a/temp_analysis/openllmetry+++ /dev/null@@ -1 +0,0 @@-Subproject commit a750944729721f6b0e4f8873a69635c5a564c9a2diff --git a/temp_analysis/pydantic-ai b/temp_analysis/pydantic-aideleted file mode 160000index babc23b3..00000000--- a/temp_analysis/pydantic-ai+++ /dev/null@@ -1 +0,0 @@-Subproject commit babc23b30a94cbfa7d6ee9cebba25309b0cf8360