Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
feat(ts-sdk): OTel message format, streaming fixes, metrics parity, new addons, and docs updates (#1005)
Aman Agarwala2f20b784bb5
diff --git a/docs/latest/sdk/integrations/azure-openai.mdx b/docs/latest/sdk/integrations/azure-openai.mdxindex 5e6e80cc..7751ee75 100644--- a/docs/latest/sdk/integrations/azure-openai.mdx+++ b/docs/latest/sdk/integrations/azure-openai.mdx@@ -9,17 +9,28 @@ OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applicatAuto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- OpenAI Python SDK client `>=1.13.0`+The integration is compatible with:+- OpenAI Python SDK client `>=1.13.0`+- OpenAI TypeScript/JavaScript SDK client `>=3.1.0` (Azure OpenAI uses the same SDK)## Get started<Steps><Step title="Install OpenLIT">Open your command line or terminal and run:- ```shell- pip install openlit- ```++ <Tabs>+ <Tab title="Python">+ ```shell+ pip install openlit+ ```+ </Tab>+ <Tab title="TypeScript">+ ```shell+ npm install openlit+ ```+ </Tab>+ </Tabs></Step><IntegrationMethods /></Steps>diff --git a/docs/latest/sdk/integrations/cohere.mdx b/docs/latest/sdk/integrations/cohere.mdxindex c86ed53f..5625c1c6 100644--- a/docs/latest/sdk/integrations/cohere.mdx+++ b/docs/latest/sdk/integrations/cohere.mdx@@ -9,8 +9,9 @@ OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applicatAuto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- Cohere Python SDK client `>= 5.3.0`+The integration is compatible with+- Cohere Python SDK client `>= 5.3.0`+- Cohere TypeScript/JavaScript SDK client `>= 7.0.0`## Get starteddiff --git a/docs/latest/sdk/integrations/google-ai-studio.mdx b/docs/latest/sdk/integrations/google-ai-studio.mdxindex 0bc99b16..663ea198 100644--- a/docs/latest/sdk/integrations/google-ai-studio.mdx+++ b/docs/latest/sdk/integrations/google-ai-studio.mdx@@ -5,21 +5,32 @@ sidebarTitle: 'Google AI Studio'import IntegrationMethods from '/snippets/integration-methods.mdx';-OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models from Google AI Studio. This includes tracking performance, token usage, costs, and how users interact with the application.+OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models from Google AI Studio (Gemini). This includes tracking performance, token usage, costs, and how users interact with the application.Auto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- google-generativeai `>= 0.7.2`+The integration is compatible with:+- Google Generative AI Python SDK `>= 1.3.0`+- Google Generative AI TypeScript/JavaScript SDK `>= 0.1.0`## Get started<Steps><Step title="Install OpenLIT">Open your command line or terminal and run:- ```shell- pip install openlit- ```++ <Tabs>+ <Tab title="Python">+ ```shell+ pip install openlit+ ```+ </Tab>+ <Tab title="TypeScript">+ ```shell+ npm install openlit+ ```+ </Tab>+ </Tabs></Step><IntegrationMethods /></Steps>@@ -45,4 +56,4 @@ The integration is compatible withhref="/latest/operator/overview">Automatically inject instrumentation into existing workloads without modifying pod specs, container images, or application code.- </Card>\ No newline at end of file+ </Card>diff --git a/docs/latest/sdk/integrations/groq.mdx b/docs/latest/sdk/integrations/groq.mdxindex 494ef555..ddc1f6d0 100644--- a/docs/latest/sdk/integrations/groq.mdx+++ b/docs/latest/sdk/integrations/groq.mdx@@ -9,17 +9,28 @@ OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applicatAuto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- Groq Python SDK client `>= 0.5.0`+The integration is compatible with:+- Groq Python SDK client `>= 0.5.0`+- Groq TypeScript/JavaScript SDK client `>= 0.5.0`## Get started<Steps><Step title="Install OpenLIT">Open your command line or terminal and run:- ```shell- pip install openlit- ```++ <Tabs>+ <Tab title="Python">+ ```shell+ pip install openlit+ ```+ </Tab>+ <Tab title="TypeScript">+ ```shell+ npm install openlit+ ```+ </Tab>+ </Tabs></Step><IntegrationMethods /></Steps>@@ -45,4 +56,4 @@ The integration is compatible withhref="/latest/operator/overview">Automatically inject instrumentation into existing workloads without modifying pod specs, container images, or application code.- </Card>\ No newline at end of file+ </Card>diff --git a/docs/latest/sdk/integrations/mistral.mdx b/docs/latest/sdk/integrations/mistral.mdxindex 859405d3..f2e5a3f2 100644--- a/docs/latest/sdk/integrations/mistral.mdx+++ b/docs/latest/sdk/integrations/mistral.mdx@@ -1,25 +1,36 @@----title: 'Monitor Mistral using OpenTelemetry'+title: 'Monitor Mistral AI using OpenTelemetry'sidebarTitle: 'Mistral'---import IntegrationMethods from '/snippets/integration-methods.mdx';-OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using Mistral models. This includes tracking performance, token usage, costs, and how users interact with the application.+OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models from Mistral AI. This includes tracking performance, token usage, costs, and how users interact with the application.Auto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- Mistral Python SDK client `>= 1.0.0`+The integration is compatible with:+- Mistral AI Python SDK client `>= 1.0.0`+- Mistral AI TypeScript/JavaScript SDK client `>= 1.0.0`## Get started<Steps><Step title="Install OpenLIT">Open your command line or terminal and run:- ```shell- pip install openlit- ```++ <Tabs>+ <Tab title="Python">+ ```shell+ pip install openlit+ ```+ </Tab>+ <Tab title="TypeScript">+ ```shell+ npm install openlit+ ```+ </Tab>+ </Tabs></Step><IntegrationMethods /></Steps>@@ -45,4 +56,4 @@ The integration is compatible withhref="/latest/operator/overview">Automatically inject instrumentation into existing workloads without modifying pod specs, container images, or application code.- </Card>\ No newline at end of file+ </Card>diff --git a/docs/latest/sdk/integrations/ollama.mdx b/docs/latest/sdk/integrations/ollama.mdxindex fd2f90f2..74757e25 100644--- a/docs/latest/sdk/integrations/ollama.mdx+++ b/docs/latest/sdk/integrations/ollama.mdx@@ -9,17 +9,28 @@ OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applicatAuto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- Ollama Python SDK client `>=0.2.0`+The integration is compatible with+- Ollama Python SDK client `>=0.2.0`+- Ollama TypeScript/JavaScript SDK client `>=0.5.0`-## Get started+## Get started<Steps><Step title="Install OpenLIT">Open your command line or terminal and run:- ```shell- pip install openlit- ```++ <Tabs>+ <Tab title="Python">+ ```shell+ pip install openlit+ ```+ </Tab>+ <Tab title="TypeScript">+ ```shell+ npm install openlit+ ```+ </Tab>+ </Tabs></Step><IntegrationMethods /></Steps>diff --git a/docs/latest/sdk/integrations/together.mdx b/docs/latest/sdk/integrations/together.mdxindex f10a9038..d1b15480 100644--- a/docs/latest/sdk/integrations/together.mdx+++ b/docs/latest/sdk/integrations/together.mdx@@ -1,32 +1,34 @@----title: 'Monitor Together using OpenTelemetry'-sidebarTitle: 'Together'+title: 'Monitor Together AI using OpenTelemetry'+sidebarTitle: 'Together AI'---import IntegrationMethods from '/snippets/integration-methods.mdx';-OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using Together models. This includes tracking performance, token usage, costs, and how users interact with the application.+OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models from Together AI. This includes tracking performance, token usage, costs, and how users interact with the application.Auto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.-The integration is compatible with-- Together Python SDK client `>=1.3.5`+The integration is compatible with:+- Together AI Python SDK `>= 1.3.5`+- Together AI TypeScript/JavaScript SDK `>= 0.1.0`## Get started<Steps><Step title="Install OpenLIT">- Open your command line or terminal and run:+ Open your command line or terminal and run:+<Tabs><Tab title="Python">```shellpip install openlit```</Tab>- <Tab title="Typescript">- ```shell- npm install openlit- ```+ <Tab title="TypeScript">+ ```shell+ npm install openlit+ ```</Tab></Tabs></Step>@@ -54,4 +56,4 @@ The integration is compatible withhref="/latest/operator/overview">Automatically inject instrumentation into existing workloads without modifying pod specs, container images, or application code.- </Card>\ No newline at end of file+ </Card>diff --git a/sdk/python/README.md b/sdk/python/README.mdindex b2964494..2f401de7 100644--- a/sdk/python/README.md+++ b/sdk/python/README.md@@ -28,7 +28,7 @@ This project proudly follows and maintains the [Semantic Conventions](https://gi- π **Auto Instrumentation**: Works with 50+ LLM providers, Agents, Vector databases, and GPUs with just one line of code.- π **OpenTelemetry-Native Observability SDKs**: Vendor-neutral SDKs that can send traces and metrics to your existing observability tool like Prometheus and Jaeger.- π² **Cost Tracking for Custom and Fine-Tuned Models**: Pass custom pricing files for accurate budgeting of custom and fine-tuned models.-- π **Suppport for OpenLIT Features**: Includes suppprt for prompt management and secrets management features available in OpenLIT.+- π **Support for OpenLIT Features**: Includes support for prompt management and secrets management features available in OpenLIT.## Auto Instrumentation Capabilitiesdiff --git a/sdk/python/tests/test_groq_trace_comparison.py b/sdk/python/tests/test_groq_trace_comparison.pynew file mode 100644index 00000000..d82a35ba--- /dev/null+++ b/sdk/python/tests/test_groq_trace_comparison.py@@ -0,0 +1,77 @@+# pylint: disable=duplicate-code+"""+Cross-Language Trace Comparison Test for Groq++This test generates a trace using Python OpenLIT and exports it for comparison+with TypeScript traces.+"""++import os+import pytest+from groq import Groq+from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor+from opentelemetry.sdk.trace import TracerProvider+import openlit+++# Initialize OpenLIT with in-memory exporter for testing+tracer_provider = TracerProvider()+console_exporter = ConsoleSpanExporter()+tracer_provider.add_span_processor(SimpleSpanProcessor(console_exporter))++openlit.init(+ environment="openlit-python-testing",+ application_name="openlit-python-groq-test",+ tracer=tracer_provider.get_tracer(__name__),+)++client = Groq(api_key=os.getenv("GROQ_API_TOKEN"))+++def test_groq_trace_structure():+ """+ Test that Groq generates traces with expected structure for comparison.+ """+ if not os.getenv("GROQ_API_TOKEN"):+ pytest.skip("GROQ_API_TOKEN not set")++ messages = [+ {+ "role": "user",+ "content": "What is LLM Observability?",+ }+ ]++ response = client.chat.completions.create(+ messages=messages,+ model="llama-3.1-8b-instant",+ max_tokens=10,+ stream=False,+ )++ assert response.object == "chat.completion"+++def test_groq_trace_metrics():+ """+ Test that Groq trace contains correct metrics.+ """+ if not os.getenv("GROQ_API_TOKEN"):+ pytest.skip("GROQ_API_TOKEN not set")++ messages = [{"role": "user", "content": "Test"}]++ response = client.chat.completions.create(+ messages=messages,+ model="llama-3.1-8b-instant",+ max_tokens=5,+ stream=False,+ )++ # Verify response has usage data+ assert hasattr(response, 'usage')+ assert hasattr(response.usage, 'prompt_tokens')+ assert hasattr(response.usage, 'completion_tokens')+ assert hasattr(response.usage, 'total_tokens')++ assert response.object == "chat.completion"diff --git a/sdk/python/tests/test_mistral_trace_comparison.py b/sdk/python/tests/test_mistral_trace_comparison.pynew file mode 100644index 00000000..f003c3c3--- /dev/null+++ b/sdk/python/tests/test_mistral_trace_comparison.py@@ -0,0 +1,73 @@+# pylint: disable=duplicate-code+"""+Cross-Language Trace Comparison Test for Mistral++This test generates a trace using Python OpenLIT and exports it for comparison+with TypeScript traces.+"""++import os+import pytest+from mistralai import Mistral+from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor+from opentelemetry.sdk.trace import TracerProvider+import openlit+++# Initialize OpenLIT with in-memory exporter for testing+tracer_provider = TracerProvider()+console_exporter = ConsoleSpanExporter()+tracer_provider.add_span_processor(SimpleSpanProcessor(console_exporter))++openlit.init(+ environment="openlit-python-testing",+ application_name="openlit-python-mistral-test",+ tracer=tracer_provider.get_tracer(__name__),+)++client = Mistral(api_key=os.getenv("MISTRAL_API_KEY"))+++def test_mistral_trace_structure():+ """+ Test that Mistral generates traces with expected structure for comparison.+ """+ if not os.getenv("MISTRAL_API_KEY"):+ pytest.skip("MISTRAL_API_KEY not set")++ messages = [+ {+ "role": "user",+ "content": "What is Mistral AI?",+ }+ ]++ response = client.chat.complete(+ model="open-mistral-7b",+ messages=messages,+ max_tokens=10,+ )++ # Verify response structure+ assert hasattr(response, 'id')+ assert hasattr(response, 'model')+ assert hasattr(response, 'choices')+ assert hasattr(response, 'usage')+++def test_mistral_embedding_trace_structure():+ """+ Test that Mistral embeddings generate traces with expected structure.+ """+ if not os.getenv("MISTRAL_API_KEY"):+ pytest.skip("MISTRAL_API_KEY not set")++ response = client.embeddings.create(+ model="mistral-embed",+ inputs="Test embedding",+ )++ # Verify response structure+ assert hasattr(response, 'model')+ assert hasattr(response, 'data')+ assert hasattr(response, 'usage')diff --git a/sdk/python/tests/trace_comparison_utils.py b/sdk/python/tests/trace_comparison_utils.pynew file mode 100644index 00000000..14b73d6a--- /dev/null+++ b/sdk/python/tests/trace_comparison_utils.py@@ -0,0 +1,178 @@+"""+Python Trace Export Utility for Cross-Language Comparison++This module provides utilities to export Python OpenLIT traces in a format+that can be compared with TypeScript traces.+"""++import json+from typing import Dict, Any+from opentelemetry.sdk.trace import ReadableSpan+from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter+++def normalize_python_span(span: ReadableSpan) -> Dict[str, Any]:+ """+ Normalize a Python OpenTelemetry span to a comparable format.++ Args:+ span: OpenTelemetry ReadableSpan from Python SDK++ Returns:+ Normalized trace dictionary matching TypeScript format+ """+ attributes = {}+ for key, value in span.attributes.items():+ # Normalize arrays+ if isinstance(value, (list, tuple)):+ attributes[key] = sorted(list(value))+ # Normalize numbers (round to 3 decimal places)+ elif isinstance(value, float):+ attributes[key] = round(value * 1000) / 1000+ elif isinstance(value, int):+ attributes[key] = value+ # Normalize strings+ elif isinstance(value, str):+ attributes[key] = value.strip()+ else:+ attributes[key] = value++ events = []+ for event in span.events:+ event_attrs = {}+ for key, value in event.attributes.items():+ if isinstance(value, (list, tuple)):+ event_attrs[key] = sorted(list(value))+ elif isinstance(value, float):+ event_attrs[key] = round(value * 1000) / 1000+ elif isinstance(value, int):+ event_attrs[key] = value+ elif isinstance(value, str):+ event_attrs[key] = value.strip()+ else:+ event_attrs[key] = value++ events.append({+ 'name': event.name,+ 'attributes': event_attrs,+ })++ # Normalize status+ from opentelemetry.trace import StatusCode # pylint: disable=import-outside-toplevel+ status_code = 'UNSET'+ if span.status.status_code == StatusCode.OK:+ status_code = 'OK'+ elif span.status.status_code == StatusCode.ERROR:+ status_code = 'ERROR'++ return {+ 'spanName': span.name,+ 'spanKind': str(span.kind),+ 'attributes': attributes,+ 'events': events,+ 'status': {+ 'code': status_code,+ 'message': span.status.description,+ },+ 'duration': span.end_time - span.start_time if span.end_time and span.start_time else None,+ }+++def export_trace_to_json(span: ReadableSpan, filepath: str) -> None:+ """+ Export a normalized trace to JSON file for comparison.++ Args:+ span: OpenTelemetry ReadableSpan+ filepath: Path to save JSON file+ """+ normalized = normalize_python_span(span)+ with open(filepath, 'w', encoding='utf-8') as f:+ json.dump(normalized, f, indent=2, default=str)+++def compare_with_typescript(+ python_span: ReadableSpan,+ typescript_trace_path: str+) -> Dict[str, Any]:+ """+ Compare Python trace with TypeScript trace from JSON file.++ Args:+ python_span: Python OpenTelemetry span+ typescript_trace_path: Path to TypeScript trace JSON file++ Returns:+ Comparison result dictionary+ """+ python_normalized = normalize_python_span(python_span)++ with open(typescript_trace_path, 'r', encoding='utf-8') as f:+ typescript_normalized = json.load(f)++ differences = []++ # Compare critical attributes+ critical_attrs = [+ 'gen_ai.system',+ 'gen_ai.operation.name',+ 'gen_ai.request.model',+ 'gen_ai.response.model',+ 'gen_ai.usage.input_tokens',+ 'gen_ai.usage.output_tokens',+ 'gen_ai.usage.total_tokens',+ ]++ for attr in critical_attrs:+ python_val = python_normalized['attributes'].get(attr)+ typescript_val = typescript_normalized['attributes'].get(attr)++ if python_val != typescript_val:+ differences.append(+ f"Attribute '{attr}': Python={python_val}, TypeScript={typescript_val}"+ )++ return {+ 'match': len(differences) == 0,+ 'differences': differences,+ 'pythonTrace': python_normalized,+ 'typescriptTrace': typescript_normalized,+ }+++def get_finished_spans(exporter: InMemorySpanExporter) -> list:+ """+ Get finished spans from an InMemorySpanExporter.++ Args:+ exporter: InMemorySpanExporter instance++ Returns:+ List of finished ReadableSpan objects+ """+ return list(exporter.get_finished_spans())+++def extract_key_metrics(span: ReadableSpan) -> Dict[str, Any]:+ """+ Extract key metrics from a span for comparison.++ Args:+ span: OpenTelemetry ReadableSpan++ Returns:+ Dictionary with key metrics+ """+ attrs = span.attributes++ return {+ 'tokens': {+ 'input': attrs.get('gen_ai.usage.input_tokens', 0),+ 'output': attrs.get('gen_ai.usage.output_tokens', 0),+ 'total': attrs.get('gen_ai.usage.total_tokens', 0),+ },+ 'cost': attrs.get('gen_ai.usage.cost', 0),+ 'model': attrs.get('gen_ai.request.model') or attrs.get('gen_ai.response.model', ''),+ 'operation': attrs.get('gen_ai.operation.name', ''),+ 'system': attrs.get('gen_ai.system', ''),+ }diff --git a/sdk/typescript/README.md b/sdk/typescript/README.mdindex a1832456..b07bf50a 100644--- a/sdk/typescript/README.md+++ b/sdk/typescript/README.md@@ -24,19 +24,25 @@ This project proudly follows and maintains the [Semantic Conventions](https://gi## β‘ Features-- π **Auto Instrumentation**: Works with 30+ LLM providers and vector databases with just one line of code.+- π **Auto Instrumentation**: Works with 8 LLM providers with just one line of code.- π **OpenTelemetry-Native Observability SDKs**: Vendor-neutral SDKs that can send traces and metrics to your existing observability tool like Prometheus and Jaeger.- π² **Cost Tracking for Custom and Fine-Tuned Models**: Pass custom pricing files for accurate budgeting of custom and fine-tuned models.-- π **Suppport for OpenLIT Features**: Includes suppprt for prompt management and secrets management features available in OpenLIT.+- π **Support for OpenLIT Features**: Includes support for prompt management and secrets management features available in OpenLIT.## Auto Instrumentation Capabilities-| LLMs |-| --------------------------------------------------------------------- |-| [β OpenAI](https://docs.openlit.io/latest/integrations/openai) | [β ChromaDB](https://docs.openlit.io/latest/integrations/chromadb) | [β LiteLLM](https://docs.openlit.io/latest/integrations/litellm) | |-| [β Anthropic](https://docs.openlit.io/latest/integrations/anthropic) |-| [β Cohere](https://docs.openlit.io/latest/integrations/cohere) |+| LLMs |+| ------------------------------------------------------------------------------------------- |+| [β OpenAI](https://docs.openlit.io/latest/integrations/openai) |+| [β Anthropic](https://docs.openlit.io/latest/integrations/anthropic) |+| [β Cohere](https://docs.openlit.io/latest/integrations/cohere) |+| [β Groq](https://docs.openlit.io/latest/integrations/groq) |+| [β Mistral](https://docs.openlit.io/latest/integrations/mistral) |+| [β Google AI Studio](https://docs.openlit.io/latest/integrations/google-ai-studio) |+| [β Together AI](https://docs.openlit.io/latest/integrations/together) |+| [β Ollama](https://docs.openlit.io/latest/integrations/ollama) |+| [β Azure OpenAI](https://docs.openlit.io/latest/integrations/azure-openai) *(via OpenAI SDK)* |## Supported Destinations@@ -56,9 +62,7 @@ This project proudly follows and maintains the [Semantic Conventions](https://gi## Supported Metrics-> **Note:** Metrics tracking in the TypeScript SDK is only available for Anthropic, Cohere, Ollama, and OpenAI integrations.--The following metrics are supported:+The following metrics are supported across all integrations:- **genaiClientUsageTokens**: Histogram for total input/output tokens used.- **genaiClientOperationDuration**: Histogram for GenAI operation duration.diff --git a/sdk/typescript/src/helpers.ts b/sdk/typescript/src/helpers.tsindex 316afd36..0434ca39 100644--- a/sdk/typescript/src/helpers.ts+++ b/sdk/typescript/src/helpers.ts@@ -8,9 +8,8 @@ export default class OpenLitHelper {try {const encoding = encodingForModel(model as TiktokenModel);return encoding.encode(text).length;- } catch (error) {- console.error(`Error in openaiTokens: ${error}`);- throw error;+ } catch {+ return OpenLitHelper.generalTokens(text);}}@@ -108,6 +107,104 @@ export default class OpenLitHelper {}}+ /**+ * Build OTel-spec input messages JSON string from provider messages array.+ * Format: [{"role": "user", "parts": [{"type": "text", "content": "..."}]}]+ */+ static buildInputMessages(messages: any[], system?: string): string {+ try {+ const otelMessages: any[] = [];++ if (system) {+ otelMessages.push({ role: 'system', parts: [{ type: 'text', content: system }] });+ }++ for (const msg of messages || []) {+ const role = msg.role || 'user';+ const content = msg.content;+ const parts: any[] = [];++ if (typeof content === 'string' && content) {+ parts.push({ type: 'text', content });+ } else if (Array.isArray(content)) {+ for (const item of content) {+ const t = item.type;+ if (t === 'text') {+ parts.push({ type: 'text', content: item.text || '' });+ } else if (t === 'image_url') {+ const url = item.image_url?.url || '';+ if (url && !url.startsWith('data:')) {+ parts.push({ type: 'uri', modality: 'image', uri: url });+ }+ } else if (t === 'image') {+ // Anthropic image format+ const url = item.source?.url || '';+ if (url && !url.startsWith('data:')) {+ parts.push({ type: 'uri', modality: 'image', uri: url });+ }+ } else if (t === 'tool_use') {+ parts.push({ type: 'tool_call', id: item.id || '', name: item.name || '', arguments: item.input || {} });+ } else if (t === 'tool_result') {+ parts.push({ type: 'tool_call_response', id: item.tool_use_id || '', response: typeof item.content === 'string' ? item.content : JSON.stringify(item.content || '') });+ }+ }+ }++ // Handle tool_calls in message (OpenAI assistant format)+ if (msg.tool_calls && Array.isArray(msg.tool_calls)) {+ for (const tc of msg.tool_calls) {+ let args = tc.function?.arguments || {};+ if (typeof args === 'string') {+ try { args = JSON.parse(args); } catch { args = { raw: args }; }+ }+ parts.push({ type: 'tool_call', id: tc.id || '', name: tc.function?.name || '', arguments: args });+ }+ }++ if (parts.length > 0) {+ otelMessages.push({ role, parts });+ }+ }++ return JSON.stringify(otelMessages);+ } catch {+ return '[]';+ }+ }++ /**+ * Build OTel-spec output messages JSON string from provider response.+ * Format: [{"role": "assistant", "parts": [{"type": "text", "content": "..."}], "finish_reason": "stop"}]+ */+ static buildOutputMessages(text: string, finishReason: string, toolCalls?: any[]): string {+ try {+ const parts: any[] = [];++ if (text) {+ parts.push({ type: 'text', content: text });+ }++ if (toolCalls && toolCalls.length > 0) {+ for (const tc of toolCalls) {+ let args = tc.function?.arguments || tc.arguments || {};+ if (typeof args === 'string') {+ try { args = JSON.parse(args); } catch { args = { raw: args }; }+ }+ parts.push({+ type: 'tool_call',+ id: tc.id || '',+ name: tc.function?.name || tc.name || '',+ arguments: args,+ });+ }+ }++ return JSON.stringify([{ role: 'assistant', parts, finish_reason: finishReason || 'stop' }]);+ } catch {+ return '[]';+ }+ }+static handleException(span: Span, error: Error): void {span.recordException(error);span.setStatus({ code: SpanStatusCode.ERROR, message: error.message });diff --git a/sdk/typescript/src/instrumentation/__tests__/anthropic-wrapper.test.ts b/sdk/typescript/src/instrumentation/__tests__/anthropic-wrapper.test.tsindex e48c2b6b..02bf6cf3 100644--- a/sdk/typescript/src/instrumentation/__tests__/anthropic-wrapper.test.ts+++ b/sdk/typescript/src/instrumentation/__tests__/anthropic-wrapper.test.ts@@ -104,7 +104,7 @@ describe('AnthropicWrapper', () => {);expect(setAttributeSpy).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,- 'stop'+ ['stop']);});});diff --git a/sdk/typescript/src/instrumentation/__tests__/google-ai-trace-comparison.test.ts b/sdk/typescript/src/instrumentation/__tests__/google-ai-trace-comparison.test.tsnew file mode 100644index 00000000..e7275763--- /dev/null+++ b/sdk/typescript/src/instrumentation/__tests__/google-ai-trace-comparison.test.ts@@ -0,0 +1,105 @@+/**+ * Cross-Language Trace Comparison Tests for Google AI Studio Integration+ */++import GoogleAIWrapper from '../google-ai/wrapper';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import BaseWrapper from '../base-wrapper';+import SemanticConvention from '../../semantic-convention';++jest.mock('../../config');+jest.mock('../../helpers');+jest.mock('../base-wrapper');++describe('Google AI Studio Cross-Language Trace Comparison', () => {+ let mockSpan: any;++ beforeEach(() => {+ // Create mock span+ mockSpan = {+ setAttribute: jest.fn(),+ addEvent: jest.fn(),+ end: jest.fn(),+ setStatus: jest.fn(),+ };++ // Mock OpenlitConfig+ (OpenlitConfig as any).environment = 'openlit-testing';+ (OpenlitConfig as any).applicationName = 'openlit-test';+ (OpenlitConfig as any).traceContent = true;+ (OpenlitConfig as any).pricing_json = {};+ (OpenlitConfig as any).updatePricingJson = jest.fn().mockResolvedValue({});++ // Mock OpenLitHelper+ (OpenLitHelper as any).getChatModelCost = jest.fn().mockReturnValue(0.001);+ (OpenLitHelper as any).handleException = jest.fn();+ (OpenLitHelper as any).createStreamProxy = jest.fn().mockImplementation((stream, generator) => stream);++ // Mock BaseWrapper+ (BaseWrapper as any).recordMetrics = jest.fn();+ (BaseWrapper as any).setBaseSpanAttributes = jest.fn().mockImplementation((span, attrs) => {+ span.setAttribute(SemanticConvention.GEN_AI_PROVIDER_NAME, attrs.aiSystem);+ span.setAttribute(SemanticConvention.GEN_AI_ENDPOINT, attrs.genAIEndpoint);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MODEL, attrs.model);+ if (attrs.cost !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_COST, attrs.cost);+ }+ });+ });++ afterEach(() => {+ jest.clearAllMocks();+ });++ describe('Generate Content Trace Consistency', () => {+ it('should set same attributes as Python SDK', async () => {+ const mockArgs = [+ {+ contents: [+ { role: 'user', parts: [{ text: 'What is Gemini?' }] },+ ],+ config: {+ temperature: 0.7,+ maxOutputTokens: 100,+ topP: 0.95,+ },+ },+ ];++ const mockResponse = {+ model: 'gemini-pro',+ candidates: [+ {+ content: {+ parts: [{ text: 'Gemini is Google\'s AI model' }],+ role: 'model',+ },+ finishReason: 'STOP',+ },+ ],+ usageMetadata: {+ promptTokenCount: 5,+ candidatesTokenCount: 10,+ totalTokenCount: 15,+ },+ };++ await GoogleAIWrapper._generateContent({+ args: mockArgs,+ genAIEndpoint: 'google.generativeai.models.generate_content',+ response: mockResponse,+ span: mockSpan,+ });++ // Verify critical attributes are set (matching Python SDK)+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_PROVIDER_NAME, 'google_ai_studio');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_MODEL, 'gemini-pro');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_MODEL, 'gemini-pro');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, 5);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, 10);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, 15);+ });+ });+});diff --git a/sdk/typescript/src/instrumentation/__tests__/groq-trace-comparison.test.ts b/sdk/typescript/src/instrumentation/__tests__/groq-trace-comparison.test.tsnew file mode 100644index 00000000..84f8e5c6--- /dev/null+++ b/sdk/typescript/src/instrumentation/__tests__/groq-trace-comparison.test.ts@@ -0,0 +1,203 @@+/**+ * Cross-Language Trace Comparison Tests for Groq Integration+ *+ * These tests verify that TypeScript and Python generate equivalent traces+ * for the same operations.+ */++import GroqWrapper from '../groq/wrapper';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import BaseWrapper from '../base-wrapper';+import SemanticConvention from '../../semantic-convention';+import {+ compareTraces,+ compareMetrics,+ createTraceValidator,+ NormalizedTrace,+} from './trace-comparison-utils';++jest.mock('../../config');+jest.mock('../../helpers');+jest.mock('../base-wrapper');++describe('Groq Cross-Language Trace Comparison', () => {+ let mockSpan: any;+ let mockTracer: any;++ beforeEach(() => {+ // Create mock span+ mockSpan = {+ setAttribute: jest.fn(),+ addEvent: jest.fn(),+ end: jest.fn(),+ setStatus: jest.fn(),+ };++ // Create mock tracer+ mockTracer = {+ startSpan: jest.fn().mockReturnValue(mockSpan),+ };++ // Mock OpenlitConfig+ (OpenlitConfig as any).environment = 'openlit-testing';+ (OpenlitConfig as any).applicationName = 'openlit-test';+ (OpenlitConfig as any).traceContent = true;+ (OpenlitConfig as any).pricing_json = {};+ (OpenlitConfig as any).updatePricingJson = jest.fn().mockResolvedValue({});++ // Mock OpenLitHelper+ (OpenLitHelper as any).getChatModelCost = jest.fn().mockReturnValue(0.001);+ (OpenLitHelper as any).openaiTokens = jest.fn().mockReturnValue(5);+ (OpenLitHelper as any).handleException = jest.fn();+ (OpenLitHelper as any).createStreamProxy = jest.fn().mockImplementation((stream, generator) => stream);++ // Mock BaseWrapper+ (BaseWrapper as any).recordMetrics = jest.fn();+ (BaseWrapper as any).setBaseSpanAttributes = jest.fn().mockImplementation((span, attrs) => {+ span.setAttribute(SemanticConvention.GEN_AI_PROVIDER_NAME, attrs.aiSystem);+ span.setAttribute(SemanticConvention.GEN_AI_ENDPOINT, attrs.genAIEndpoint);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MODEL, attrs.model);+ if (attrs.cost !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_COST, attrs.cost);+ }+ });+ });++ afterEach(() => {+ jest.clearAllMocks();+ });++ describe('Chat Completion Trace Consistency', () => {+ it('should set same attributes as Python SDK', async () => {+ const mockArgs = [+ {+ messages: [{ role: 'user', content: 'What is LLM Observability?' }],+ model: 'llama-3.1-8b-instant',+ max_tokens: 100,+ temperature: 0.7,+ stream: false,+ },+ ];++ const mockResponse = {+ id: 'test-id-123',+ created: Date.now(),+ model: 'llama-3.1-8b-instant',+ choices: [+ {+ index: 0,+ finish_reason: 'stop',+ message: { role: 'assistant', content: 'LLM Observability is...' },+ },+ ],+ usage: {+ prompt_tokens: 10,+ completion_tokens: 20,+ total_tokens: 30,+ },+ };++ await GroqWrapper._chatCompletion({+ args: mockArgs,+ genAIEndpoint: 'groq.chat.completions',+ response: mockResponse,+ span: mockSpan,+ });++ // Verify critical attributes are set (matching Python SDK)+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_PROVIDER_NAME, 'groq');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_ENDPOINT, 'groq.chat.completions');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_MODEL, 'llama-3.1-8b-instant');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_MODEL, 'llama-3.1-8b-instant');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_ID, 'test-id-123');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, 10);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, 20);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, 30);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, 0.7);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, 100);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, false);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, ['stop']);+ });++ it('should set streaming attributes matching Python SDK', async () => {+ const mockArgs = [+ {+ messages: [{ role: 'user', content: 'Test streaming' }],+ model: 'llama-3.1-8b-instant',+ stream: true,+ },+ ];++ // Mock streaming response+ async function* mockStream() {+ yield {+ id: 'test-id',+ created: Date.now(),+ model: 'llama-3.1-8b-instant',+ choices: [{ delta: { content: 'Hello' } }],+ };+ yield {+ id: 'test-id',+ created: Date.now(),+ model: 'llama-3.1-8b-instant',+ choices: [{ delta: { content: ' world' } }],+ };+ }++ const generator = GroqWrapper._chatCompletionGenerator({+ args: mockArgs,+ genAIEndpoint: 'groq.chat.completions',+ response: mockStream(),+ span: mockSpan,+ });++ // Consume the stream+ for await (const _ of generator) {+ // Consume chunks+ }++ // Verify streaming-specific attributes are set+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, true);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_PROVIDER_NAME, 'groq');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);+ });+ });++ describe('Trace Validation', () => {+ it('should set all required attributes matching Python SDK', async () => {+ const mockArgs = [+ {+ messages: [{ role: 'user', content: 'Test' }],+ model: 'llama-3.1-8b-instant',+ },+ ];++ const mockResponse = {+ id: 'test',+ model: 'llama-3.1-8b-instant',+ choices: [{ message: { content: 'Response' }, finish_reason: 'stop' }],+ usage: { prompt_tokens: 5, completion_tokens: 10, total_tokens: 15 },+ };++ await GroqWrapper._chatCompletion({+ args: mockArgs,+ genAIEndpoint: 'groq.chat.completions',+ response: mockResponse,+ span: mockSpan,+ });++ // Verify all required attributes are set (matching Python expectations)+ const setAttributeCalls = (mockSpan.setAttribute as jest.Mock).mock.calls;+ const attributeMap = new Map(setAttributeCalls.map(([key, value]) => [key, value]));++ expect(attributeMap.get(SemanticConvention.GEN_AI_PROVIDER_NAME)).toBe('groq');+ expect(attributeMap.get(SemanticConvention.GEN_AI_OPERATION)).toBe(SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);+ expect(attributeMap.get(SemanticConvention.GEN_AI_REQUEST_MODEL)).toBe('llama-3.1-8b-instant');+ expect(attributeMap.get(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS)).toBe(5);+ expect(attributeMap.get(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS)).toBe(10);+ expect(attributeMap.get(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS)).toBe(15);+ });+ });+});diff --git a/sdk/typescript/src/instrumentation/__tests__/mistral-trace-comparison.test.ts b/sdk/typescript/src/instrumentation/__tests__/mistral-trace-comparison.test.tsnew file mode 100644index 00000000..174b33ef--- /dev/null+++ b/sdk/typescript/src/instrumentation/__tests__/mistral-trace-comparison.test.ts@@ -0,0 +1,139 @@+/**+ * Cross-Language Trace Comparison Tests for Mistral Integration+ */++import MistralWrapper from '../mistral/wrapper';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import BaseWrapper from '../base-wrapper';+import SemanticConvention from '../../semantic-convention';++jest.mock('../../config');+jest.mock('../../helpers');+jest.mock('../base-wrapper');++describe('Mistral Cross-Language Trace Comparison', () => {+ let mockSpan: any;++ beforeEach(() => {+ // Create mock span+ mockSpan = {+ setAttribute: jest.fn(),+ addEvent: jest.fn(),+ end: jest.fn(),+ setStatus: jest.fn(),+ };++ // Mock OpenlitConfig+ (OpenlitConfig as any).environment = 'openlit-testing';+ (OpenlitConfig as any).applicationName = 'openlit-test';+ (OpenlitConfig as any).traceContent = true;+ (OpenlitConfig as any).pricing_json = {};+ (OpenlitConfig as any).updatePricingJson = jest.fn().mockResolvedValue({});++ // Mock OpenLitHelper+ (OpenLitHelper as any).getChatModelCost = jest.fn().mockReturnValue(0.001);+ (OpenLitHelper as any).getEmbedModelCost = jest.fn().mockReturnValue(0.0001);+ (OpenLitHelper as any).openaiTokens = jest.fn().mockReturnValue(5);+ (OpenLitHelper as any).handleException = jest.fn();+ (OpenLitHelper as any).createStreamProxy = jest.fn().mockImplementation((stream, generator) => stream);++ // Mock BaseWrapper+ (BaseWrapper as any).recordMetrics = jest.fn();+ (BaseWrapper as any).setBaseSpanAttributes = jest.fn().mockImplementation((span, attrs) => {+ span.setAttribute(SemanticConvention.GEN_AI_PROVIDER_NAME, attrs.aiSystem);+ span.setAttribute(SemanticConvention.GEN_AI_ENDPOINT, attrs.genAIEndpoint);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MODEL, attrs.model);+ if (attrs.cost !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_COST, attrs.cost);+ }+ });+ });++ afterEach(() => {+ jest.clearAllMocks();+ });++ describe('Chat Completion Trace Consistency', () => {+ it('should set same attributes as Python SDK', async () => {+ const mockArgs = [+ {+ messages: [{ role: 'user', content: 'What is Mistral AI?' }],+ model: 'mistral-small-latest',+ max_tokens: 50,+ temperature: 0.7,+ stream: false,+ },+ ];++ const mockResponse = {+ id: 'mistral-test-id',+ created: Date.now(),+ model: 'mistral-small-latest',+ choices: [+ {+ index: 0,+ finish_reason: 'stop',+ message: { role: 'assistant', content: 'Mistral AI is...' },+ },+ ],+ usage: {+ prompt_tokens: 8,+ completion_tokens: 15,+ total_tokens: 23,+ },+ };++ await MistralWrapper._chatCompletion({+ args: mockArgs,+ genAIEndpoint: 'mistral.chat.completions',+ response: mockResponse,+ span: mockSpan,+ });++ // Verify critical attributes are set (matching Python SDK)+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_PROVIDER_NAME, 'mistral');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_MODEL, 'mistral-small-latest');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_MODEL, 'mistral-small-latest');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, 8);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, 15);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, 23);+ });+ });++ describe('Embedding Trace Consistency', () => {+ it('should set embedding attributes matching Python SDK', async () => {+ const mockArgs = [+ {+ model: 'mistral-embed',+ input: 'Test embedding text',+ },+ ];++ const mockResponse = {+ model: 'mistral-embed',+ data: [{ embedding: [0.1, 0.2, 0.3] }],+ usage: {+ prompt_tokens: 3,+ total_tokens: 3,+ },+ };++ const mockTracer: any = {+ startSpan: jest.fn().mockReturnValue(mockSpan),+ };++ const patchMethod = MistralWrapper._patchEmbedding(mockTracer);+ const wrappedMethod = patchMethod(async () => mockResponse);++ await wrappedMethod.call({}, ...mockArgs);++ // Verify embedding-specific attributes+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_PROVIDER_NAME, 'mistral');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_EMBEDDING);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_MODEL, 'mistral-embed');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, 3);+ });+ });+});diff --git a/sdk/typescript/src/instrumentation/__tests__/together-trace-comparison.test.ts b/sdk/typescript/src/instrumentation/__tests__/together-trace-comparison.test.tsnew file mode 100644index 00000000..4fa80660--- /dev/null+++ b/sdk/typescript/src/instrumentation/__tests__/together-trace-comparison.test.ts@@ -0,0 +1,104 @@+/**+ * Cross-Language Trace Comparison Tests for Together AI Integration+ */++import TogetherWrapper from '../together/wrapper';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import BaseWrapper from '../base-wrapper';+import SemanticConvention from '../../semantic-convention';++jest.mock('../../config');+jest.mock('../../helpers');+jest.mock('../base-wrapper');++describe('Together AI Cross-Language Trace Comparison', () => {+ let mockSpan: any;++ beforeEach(() => {+ // Create mock span+ mockSpan = {+ setAttribute: jest.fn(),+ addEvent: jest.fn(),+ end: jest.fn(),+ setStatus: jest.fn(),+ };++ // Mock OpenlitConfig+ (OpenlitConfig as any).environment = 'openlit-testing';+ (OpenlitConfig as any).applicationName = 'openlit-test';+ (OpenlitConfig as any).traceContent = true;+ (OpenlitConfig as any).pricing_json = {};+ (OpenlitConfig as any).updatePricingJson = jest.fn().mockResolvedValue({});++ // Mock OpenLitHelper+ (OpenLitHelper as any).getChatModelCost = jest.fn().mockReturnValue(0.001);+ (OpenLitHelper as any).openaiTokens = jest.fn().mockReturnValue(5);+ (OpenLitHelper as any).handleException = jest.fn();+ (OpenLitHelper as any).createStreamProxy = jest.fn().mockImplementation((stream, generator) => stream);++ // Mock BaseWrapper+ (BaseWrapper as any).recordMetrics = jest.fn();+ (BaseWrapper as any).setBaseSpanAttributes = jest.fn().mockImplementation((span, attrs) => {+ span.setAttribute(SemanticConvention.GEN_AI_PROVIDER_NAME, attrs.aiSystem);+ span.setAttribute(SemanticConvention.GEN_AI_ENDPOINT, attrs.genAIEndpoint);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MODEL, attrs.model);+ if (attrs.cost !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_COST, attrs.cost);+ }+ });+ });++ afterEach(() => {+ jest.clearAllMocks();+ });++ describe('Chat Completion Trace Consistency', () => {+ it('should set same attributes as Python SDK', async () => {+ const mockArgs = [+ {+ messages: [{ role: 'user', content: 'What is Together AI?' }],+ model: 'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo',+ max_tokens: 50,+ temperature: 0.7,+ top_p: 0.9,+ stream: false,+ },+ ];++ const mockResponse = {+ id: 'together-test-id',+ created: Date.now(),+ model: 'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo',+ choices: [+ {+ index: 0,+ finish_reason: 'stop',+ message: { role: 'assistant', content: 'Together AI is...' },+ },+ ],+ usage: {+ prompt_tokens: 9,+ completion_tokens: 12,+ total_tokens: 21,+ },+ };++ await TogetherWrapper._chatCompletion({+ args: mockArgs,+ genAIEndpoint: 'together.chat.completions',+ response: mockResponse,+ span: mockSpan,+ });++ // Verify critical attributes are set (matching Python SDK)+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_PROVIDER_NAME, 'together');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_REQUEST_MODEL, 'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_RESPONSE_MODEL, 'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo');+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, 9);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, 12);+ expect(mockSpan.setAttribute).toHaveBeenCalledWith(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, 21);+ });+ });+});diff --git a/sdk/typescript/src/instrumentation/__tests__/trace-comparison-utils.ts b/sdk/typescript/src/instrumentation/__tests__/trace-comparison-utils.tsnew file mode 100644index 00000000..19378764--- /dev/null+++ b/sdk/typescript/src/instrumentation/__tests__/trace-comparison-utils.ts@@ -0,0 +1,340 @@+/**+ * Cross-Language Trace Comparison Utilities+ *+ * This module provides utilities to compare traces generated by Python and TypeScript+ * OpenLIT SDKs to ensure consistency across implementations.+ */++import { Span, SpanStatusCode } from '@opentelemetry/api';+import { ReadableSpan } from '@opentelemetry/sdk-trace-base';++export interface NormalizedTrace {+ spanName: string;+ spanKind: string;+ attributes: Record<string, any>;+ events: Array<{ name: string; attributes?: Record<string, any> }>;+ status: { code: string; message?: string };+ duration?: number;+}++export interface TraceComparisonResult {+ match: boolean;+ differences: string[];+ pythonTrace?: NormalizedTrace;+ typescriptTrace?: NormalizedTrace;+}++/**+ * Normalize a TypeScript span to a comparable format+ */+export function normalizeTypeScriptSpan(span: ReadableSpan | Span): NormalizedTrace {+ const spanData = span as any;++ return {+ spanName: spanData.name || '',+ spanKind: spanData.kind?.toString() || 'INTERNAL',+ attributes: normalizeAttributes(spanData.attributes || {}),+ events: (spanData.events || []).map((event: any) => ({+ name: event.name || '',+ attributes: normalizeAttributes(event.attributes || {}),+ })),+ status: {+ code: spanData.status?.code === SpanStatusCode.OK ? 'OK' :+ spanData.status?.code === SpanStatusCode.ERROR ? 'ERROR' : 'UNSET',+ message: spanData.status?.message,+ },+ duration: spanData.duration?.[0] ? spanData.duration[0] * 1e9 + spanData.duration[1] : undefined,+ };+}++/**+ * Normalize attributes for comparison (handles array/object differences)+ */+function normalizeAttributes(attrs: Record<string, any>): Record<string, any> {+ const normalized: Record<string, any> = {};++ for (const [key, value] of Object.entries(attrs)) {+ // Normalize arrays - sort for comparison+ if (Array.isArray(value)) {+ normalized[key] = [...value].sort();+ }+ // Normalize numbers (handle float precision)+ else if (typeof value === 'number') {+ normalized[key] = Math.round(value * 1000) / 1000; // Round to 3 decimal places+ }+ // Normalize strings (trim whitespace)+ else if (typeof value === 'string') {+ normalized[key] = value.trim();+ }+ // Keep other types as-is+ else {+ normalized[key] = value;+ }+ }++ return normalized;+}++/**+ * Compare two normalized traces+ */+export function compareTraces(+ pythonTrace: NormalizedTrace,+ typescriptTrace: NormalizedTrace+): TraceComparisonResult {+ const differences: string[] = [];++ // Compare span names+ if (pythonTrace.spanName !== typescriptTrace.spanName) {+ differences.push(+ `Span name mismatch: Python="${pythonTrace.spanName}", TypeScript="${typescriptTrace.spanName}"`+ );+ }++ // Compare span kind+ if (pythonTrace.spanKind !== typescriptTrace.spanKind) {+ differences.push(+ `Span kind mismatch: Python="${pythonTrace.spanKind}", TypeScript="${typescriptTrace.spanKind}"`+ );+ }++ // Compare status+ if (pythonTrace.status.code !== typescriptTrace.status.code) {+ differences.push(+ `Status code mismatch: Python="${pythonTrace.status.code}", TypeScript="${typescriptTrace.status.code}"`+ );+ }++ // Compare attributes+ const attrDiff = compareAttributes(pythonTrace.attributes, typescriptTrace.attributes);+ differences.push(...attrDiff);++ // Compare events (optional - events may differ slightly)+ const eventDiff = compareEvents(pythonTrace.events, typescriptTrace.events);+ if (eventDiff.length > 0) {+ differences.push(`Event differences: ${eventDiff.join(', ')}`);+ }++ return {+ match: differences.length === 0,+ differences,+ pythonTrace,+ typescriptTrace,+ };+}++/**+ * Compare attributes between two traces+ */+function compareAttributes(+ pythonAttrs: Record<string, any>,+ typescriptAttrs: Record<string, any>+): string[] {+ const differences: string[] = [];+ const allKeys = new Set([...Object.keys(pythonAttrs), ...Object.keys(typescriptAttrs)]);++ // Critical attributes that must match+ const criticalAttributes = [+ 'gen_ai.system',+ 'gen_ai.operation.name',+ 'gen_ai.request.model',+ 'gen_ai.response.model',+ 'gen_ai.usage.input_tokens',+ 'gen_ai.usage.output_tokens',+ 'gen_ai.usage.total_tokens',+ 'gen_ai.usage.cost',+ 'gen_ai.endpoint',+ 'gen_ai.environment',+ 'gen_ai.application_name',+ ];++ for (const key of allKeys) {+ const pythonValue = pythonAttrs[key];+ const typescriptValue = typescriptAttrs[key];++ const isCritical = criticalAttributes.includes(key);++ if (pythonValue === undefined && typescriptValue !== undefined) {+ if (isCritical) {+ differences.push(`Missing in Python: ${key} (TypeScript has: ${typescriptValue})`);+ }+ } else if (typescriptValue === undefined && pythonValue !== undefined) {+ if (isCritical) {+ differences.push(`Missing in TypeScript: ${key} (Python has: ${pythonValue})`);+ }+ } else if (pythonValue !== typescriptValue) {+ // For critical attributes, always report differences+ if (isCritical) {+ differences.push(+ `Attribute "${key}" mismatch: Python="${pythonValue}", TypeScript="${typescriptValue}"`+ );+ }+ // For non-critical, only report if significantly different+ else if (typeof pythonValue === 'number' && typeof typescriptValue === 'number') {+ const diff = Math.abs(pythonValue - typescriptValue);+ if (diff > 0.01) { // Allow small floating point differences+ differences.push(+ `Attribute "${key}" significant difference: Python=${pythonValue}, TypeScript=${typescriptValue}`+ );+ }+ }+ }+ }++ return differences;+}++/**+ * Compare events between two traces+ */+function compareEvents(+ pythonEvents: Array<{ name: string; attributes?: Record<string, any> }>,+ typescriptEvents: Array<{ name: string; attributes?: Record<string, any> }>+): string[] {+ const differences: string[] = [];++ if (pythonEvents.length !== typescriptEvents.length) {+ differences.push(+ `Event count mismatch: Python=${pythonEvents.length}, TypeScript=${typescriptEvents.length}`+ );+ }++ // Compare event names (order may differ)+ const pythonEventNames = pythonEvents.map(e => e.name).sort();+ const typescriptEventNames = typescriptEvents.map(e => e.name).sort();++ if (JSON.stringify(pythonEventNames) !== JSON.stringify(typescriptEventNames)) {+ differences.push(+ `Event names differ: Python=[${pythonEventNames.join(', ')}], TypeScript=[${typescriptEventNames.join(', ')}]`+ );+ }++ return differences;+}++/**+ * Extract key metrics from a trace for comparison+ */+export function extractKeyMetrics(trace: NormalizedTrace): {+ tokens: { input: number; output: number; total: number };+ cost: number;+ model: string;+ operation: string;+ system: string;+} {+ const attrs = trace.attributes;++ return {+ tokens: {+ input: attrs['gen_ai.usage.input_tokens'] || 0,+ output: attrs['gen_ai.usage.output_tokens'] || 0,+ total: attrs['gen_ai.usage.total_tokens'] || 0,+ },+ cost: attrs['gen_ai.usage.cost'] || 0,+ model: attrs['gen_ai.request.model'] || attrs['gen_ai.response.model'] || '',+ operation: attrs['gen_ai.operation.name'] || '',+ system: attrs['gen_ai.system'] || '',+ };+}++/**+ * Compare key metrics between Python and TypeScript traces+ */+export function compareMetrics(+ pythonTrace: NormalizedTrace,+ typescriptTrace: NormalizedTrace+): { match: boolean; differences: string[] } {+ const pythonMetrics = extractKeyMetrics(pythonTrace);+ const typescriptMetrics = extractKeyMetrics(typescriptTrace);+ const differences: string[] = [];++ // Compare tokens (allow small differences due to estimation)+ const tokenTolerance = 5; // Allow 5 token difference+ if (Math.abs(pythonMetrics.tokens.input - typescriptMetrics.tokens.input) > tokenTolerance) {+ differences.push(+ `Input tokens differ significantly: Python=${pythonMetrics.tokens.input}, TypeScript=${typescriptMetrics.tokens.input}`+ );+ }+ if (Math.abs(pythonMetrics.tokens.output - typescriptMetrics.tokens.output) > tokenTolerance) {+ differences.push(+ `Output tokens differ significantly: Python=${pythonMetrics.tokens.output}, TypeScript=${typescriptMetrics.tokens.output}`+ );+ }++ // Compare cost (allow small differences due to rounding)+ const costTolerance = 0.0001;+ if (Math.abs(pythonMetrics.cost - typescriptMetrics.cost) > costTolerance) {+ differences.push(+ `Cost differs: Python=${pythonMetrics.cost}, TypeScript=${typescriptMetrics.cost}`+ );+ }++ // Compare model (must match exactly)+ if (pythonMetrics.model !== typescriptMetrics.model) {+ differences.push(+ `Model mismatch: Python="${pythonMetrics.model}", TypeScript="${typescriptMetrics.model}"`+ );+ }++ // Compare operation (must match exactly)+ if (pythonMetrics.operation !== typescriptMetrics.operation) {+ differences.push(+ `Operation mismatch: Python="${pythonMetrics.operation}", TypeScript="${typescriptMetrics.operation}"`+ );+ }++ // Compare system (must match exactly)+ if (pythonMetrics.system !== typescriptMetrics.system) {+ differences.push(+ `System mismatch: Python="${pythonMetrics.system}", TypeScript="${typescriptMetrics.system}"`+ );+ }++ return {+ match: differences.length === 0,+ differences,+ };+}++/**+ * Create a test helper that validates trace consistency+ */+export function createTraceValidator(+ providerName: string,+ expectedAttributes: string[] = []+) {+ return {+ validateTrace: (trace: NormalizedTrace) => {+ const errors: string[] = [];++ // Check required attributes+ for (const attr of expectedAttributes) {+ if (!(attr in trace.attributes)) {+ errors.push(`Missing required attribute: ${attr}`);+ }+ }++ // Check system matches provider+ if (trace.attributes['gen_ai.system'] !== providerName) {+ errors.push(+ `System mismatch: expected "${providerName}", got "${trace.attributes['gen_ai.system']}"`+ );+ }++ // Check operation is set+ if (!trace.attributes['gen_ai.operation.name']) {+ errors.push('Missing operation name');+ }++ // Check tokens are present+ if (!trace.attributes['gen_ai.usage.input_tokens'] && !trace.attributes['gen_ai.usage.output_tokens']) {+ errors.push('Missing token usage information');+ }++ return {+ valid: errors.length === 0,+ errors,+ };+ },+ };+}diff --git a/sdk/typescript/src/instrumentation/anthropic/index.ts b/sdk/typescript/src/instrumentation/anthropic/index.tsindex dfe3d97e..c5e40a97 100644--- a/sdk/typescript/src/instrumentation/anthropic/index.ts+++ b/sdk/typescript/src/instrumentation/anthropic/index.ts@@ -39,12 +39,13 @@ export default class OpenlitAnthropicInstrumentation extends InstrumentationBaseprotected _patch(moduleExports: typeof Anthropic) {try {- if (isWrapped(moduleExports.Anthropic.Messages.prototype.create)) {- this._unwrap(moduleExports.Anthropic.Messages.prototype, 'create');+ const AnthropicClass = (moduleExports as any).Anthropic ?? moduleExports;+ if (isWrapped(AnthropicClass.Messages.prototype.create)) {+ this._unwrap(AnthropicClass.Messages.prototype, 'create');}this._wrap(- moduleExports.Anthropic.Messages.prototype,+ AnthropicClass.Messages.prototype,'create',AnthropicWrapper._patchMessageCreate(this.tracer));@@ -54,6 +55,7 @@ export default class OpenlitAnthropicInstrumentation extends InstrumentationBase}protected _unpatch(moduleExports: typeof Anthropic) {- this._unwrap(moduleExports.Anthropic.Messages.prototype, 'create');+ const AnthropicClass = (moduleExports as any).Anthropic ?? moduleExports;+ this._unwrap(AnthropicClass.Messages.prototype, 'create');}}diff --git a/sdk/typescript/src/instrumentation/anthropic/wrapper.ts b/sdk/typescript/src/instrumentation/anthropic/wrapper.tsindex 28d6ecdf..b1fd876b 100644--- a/sdk/typescript/src/instrumentation/anthropic/wrapper.ts+++ b/sdk/typescript/src/instrumentation/anthropic/wrapper.ts@@ -86,6 +86,7 @@ export default class AnthropicWrapper extends BaseWrapper {response: any;span: Span;}) {+ let metricParams: BaseSpanAttributes | undefined;try {const result = {id: '0',@@ -139,7 +140,7 @@ export default class AnthropicWrapper extends BaseWrapper {result.usage.total_tokens = result.usage.output_tokens + result.usage.input_tokens;- await AnthropicWrapper._messageCreateCommonSetter({+ metricParams = await AnthropicWrapper._messageCreateCommonSetter({args,genAIEndpoint,result,@@ -150,6 +151,9 @@ export default class AnthropicWrapper extends BaseWrapper {OpenLitHelper.handleException(span, e);} finally {span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }}}@@ -174,34 +178,8 @@ export default class AnthropicWrapper extends BaseWrapper {top_k,user,stream = false,- stop_reason,} = args[0];- // Format 'messages' into a single string- const messagePrompt = messages || '';- const formattedMessages = [];-- for (const message of messagePrompt) {- const role = message.role;- const content = message.content;-- if (Array.isArray(content)) {- const contentStr = content- .map((item) => {- if ('type' in item) {- return `${item.type}: ${item.text ? item.text : item.image_url}`;- } else {- return `text: ${item.text}`;- }- })- .join(', ');- formattedMessages.push(`${role}: ${contentStr}`);- } else {- formattedMessages.push(`${role}: ${content}`);- }- }-- const prompt = formattedMessages.join('\n');span.setAttribute(SemanticConvention.GEN_AI_OPERATION,SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT@@ -228,17 +206,19 @@ export default class AnthropicWrapper extends BaseWrapper {aiSystem: AnthropicWrapper.aiSystem,});+ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, model);+// Request Params attributes : Startspan.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, top_p);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_K, top_k);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, max_tokens);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);- span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, stop_reason);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_SEED, seed);if (traceContent) {- span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt);+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages || []));}// Request Params attributes : End@@ -248,16 +228,15 @@ export default class AnthropicWrapper extends BaseWrapper {SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS,result.usage.input_tokens + result.usage.output_tokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, result.usage.input_tokens + result.usage.output_tokens);if (result.stop_reason) {- span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, result.stop_reason);+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [result.stop_reason]);}if (traceContent) {- span.setAttribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES,- result.content?.[0]?.text || ''- );+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(result.content?.[0]?.text || '', result.stop_reason || 'stop'));}return {genAIEndpoint,diff --git a/sdk/typescript/src/instrumentation/cohere/wrapper.ts b/sdk/typescript/src/instrumentation/cohere/wrapper.tsindex 6570aa34..98168748 100644--- a/sdk/typescript/src/instrumentation/cohere/wrapper.ts+++ b/sdk/typescript/src/instrumentation/cohere/wrapper.ts@@ -239,7 +239,7 @@ export default class CohereWrapper extends BaseWrapper {span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);if (traceContent) {- span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, message);+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages([{ role: 'user', content: message }]));}// Request Params attributes : End@@ -265,6 +265,9 @@ export default class CohereWrapper extends BaseWrapper {aiSystem: CohereWrapper.aiSystem,});+ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, model);+span.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS,result.meta.billedUnits.inputTokens@@ -277,17 +280,17 @@ export default class CohereWrapper extends BaseWrapper {SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS,result.meta.billedUnits.inputTokens + result.meta.billedUnits.outputTokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, result.meta.billedUnits.inputTokens + result.meta.billedUnits.outputTokens);if (result.finishReason) {- span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, result.finishReason);+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [result.finishReason]);}- if (tools) {- span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, 'Function called with tools');- } else {- if (traceContent) {- span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, result.text);- }+ if (traceContent) {+ span.setAttribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(result.text || '', result.finishReason || 'stop')+ );}// Return metric parameters instead of recording metrics directlydiff --git a/sdk/typescript/src/instrumentation/google-ai/index.ts b/sdk/typescript/src/instrumentation/google-ai/index.tsnew file mode 100644index 00000000..56439fc7--- /dev/null+++ b/sdk/typescript/src/instrumentation/google-ai/index.ts@@ -0,0 +1,68 @@+import {+ InstrumentationBase,+ InstrumentationModuleDefinition,+ InstrumentationNodeModuleDefinition,+ isWrapped,+} from '@opentelemetry/instrumentation';+import { InstrumentationConfig } from '@opentelemetry/instrumentation';+import { INSTRUMENTATION_PREFIX } from '../../constant';+import GoogleAIWrapper from './wrapper';++export interface GoogleAIInstrumentationConfig extends InstrumentationConfig {}++export default class OpenlitGoogleAIInstrumentation extends InstrumentationBase {+ constructor(config: GoogleAIInstrumentationConfig = {}) {+ super(`${INSTRUMENTATION_PREFIX}/instrumentation-google-ai`, '1.0.0', config);+ }++ protected init(): void | InstrumentationModuleDefinition | InstrumentationModuleDefinition[] {+ const module = new InstrumentationNodeModuleDefinition(+ '@google/generative-ai',+ ['>=0.1.0'],+ (moduleExports) => {+ this._patch(moduleExports);+ return moduleExports;+ },+ (moduleExports) => {+ if (moduleExports !== undefined) {+ this._unpatch(moduleExports);+ }+ }+ );+ return [module];+ }++ public manualPatch(googleAI: any): void {+ this._patch(googleAI);+ }++ protected _patch(moduleExports: any) {+ try {+ if (isWrapped(moduleExports.GenerativeModel.prototype.generateContent)) {+ this._unwrap(moduleExports.GenerativeModel.prototype, 'generateContent');+ }+ if (isWrapped(moduleExports.GenerativeModel.prototype.generateContentStream)) {+ this._unwrap(moduleExports.GenerativeModel.prototype, 'generateContentStream');+ }++ this._wrap(+ moduleExports.GenerativeModel.prototype,+ 'generateContent',+ GoogleAIWrapper._patchGenerateContent(this.tracer)+ );++ this._wrap(+ moduleExports.GenerativeModel.prototype,+ 'generateContentStream',+ GoogleAIWrapper._patchGenerateContent(this.tracer)+ );+ } catch (e) {+ console.error('Error in _patch method:', e);+ }+ }++ protected _unpatch(moduleExports: any) {+ this._unwrap(moduleExports.GenerativeModel.prototype, 'generateContent');+ this._unwrap(moduleExports.GenerativeModel.prototype, 'generateContentStream');+ }+}diff --git a/sdk/typescript/src/instrumentation/google-ai/wrapper.ts b/sdk/typescript/src/instrumentation/google-ai/wrapper.tsnew file mode 100644index 00000000..723af413--- /dev/null+++ b/sdk/typescript/src/instrumentation/google-ai/wrapper.ts@@ -0,0 +1,319 @@+import { Span, SpanKind, Tracer, context, trace } from '@opentelemetry/api';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import SemanticConvention from '../../semantic-convention';+import BaseWrapper from '../base-wrapper';++class GoogleAIWrapper extends BaseWrapper {+ static aiSystem = 'google_ai_studio';++ static _patchGenerateContent(tracer: Tracer): any {+ const genAIEndpoint = 'google.generativeai.models.generate_content';+ return (originalMethod: (...args: any[]) => any) => {+ return async function (this: any, ...args: any[]) {+ const span = tracer.startSpan(genAIEndpoint, { kind: SpanKind.CLIENT });+ return context+ .with(trace.setSpan(context.active(), span), async () => {+ return originalMethod.apply(this, args);+ })+ .then((response: any) => {+ // generateContentStream returns { stream, response } β stream is on .stream+ if (response && response.stream && typeof response.stream[Symbol.asyncIterator] === 'function') {+ const wrappedStream = GoogleAIWrapper._generateContentStreamGenerator({+ args,+ genAIEndpoint,+ response: response.stream,+ span,+ });+ return { ...response, stream: wrappedStream };+ }++ return GoogleAIWrapper._generateContent({ args, genAIEndpoint, response, span });+ })+ .catch((e: any) => {+ OpenLitHelper.handleException(span, e);+ span.end();+ throw e;+ });+ };+ };+ }++ static async _generateContent({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): Promise<any> {+ let metricParams;+ try {+ metricParams = await GoogleAIWrapper._generateContentCommonSetter({+ args,+ genAIEndpoint,+ result: response,+ span,+ });+ return response;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async *_generateContentStreamGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): AsyncGenerator<unknown, any, unknown> {+ let metricParams;+ const timestamps: number[] = [];+ const startTime = Date.now();++ try {+ const result = {+ model: '',+ text: '',+ candidates: [] as any[],+ usageMetadata: {+ promptTokenCount: 0,+ candidatesTokenCount: 0,+ totalTokenCount: 0,+ },+ };++ for await (const chunk of response) {+ timestamps.push(Date.now());++ if (chunk.modelVersion || chunk.model) {+ result.model = chunk.modelVersion || chunk.model;+ }++ const chunkText = typeof chunk.text === 'function' ? chunk.text() : chunk.text;+ if (chunkText) {+ result.text += chunkText;+ }++ if (chunk.candidates && chunk.candidates.length > 0) {+ if (result.candidates.length === 0) {+ result.candidates = chunk.candidates.map((c: any) => ({+ content: { parts: [{ text: '' }], role: 'model' },+ finishReason: c.finishReason || '',+ safetyRatings: c.safetyRatings || [],+ }));+ }++ chunk.candidates.forEach((c: any, idx: number) => {+ if (c.content?.parts) {+ c.content.parts.forEach((part: any) => {+ if (part.text) {+ result.candidates[idx].content.parts[0].text += part.text;+ }+ });+ }+ if (c.finishReason) {+ result.candidates[idx].finishReason = c.finishReason;+ }+ });+ }++ if (chunk.usageMetadata) {+ result.usageMetadata = {+ promptTokenCount: chunk.usageMetadata.promptTokenCount || result.usageMetadata.promptTokenCount,+ candidatesTokenCount: chunk.usageMetadata.candidatesTokenCount || result.usageMetadata.candidatesTokenCount,+ totalTokenCount: chunk.usageMetadata.totalTokenCount || result.usageMetadata.totalTokenCount,+ };+ }++ yield chunk;+ }++ // Calculate TTFT and TBT+ const ttft = timestamps.length > 0 ? (timestamps[0] - startTime) / 1000 : 0;+ let tbt = 0;+ if (timestamps.length > 1) {+ const timeDiffs = timestamps.slice(1).map((t, i) => t - timestamps[i]);+ tbt = timeDiffs.reduce((a, b) => a + b, 0) / timeDiffs.length / 1000;+ }++ metricParams = await GoogleAIWrapper._generateContentCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft,+ tbt,+ isStream: true,+ });++ return result;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async _generateContentCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft = 0,+ tbt = 0,+ isStream = false,+ }: {+ args: any[];+ genAIEndpoint: string;+ result: any;+ span: Span;+ ttft?: number;+ tbt?: number;+ isStream?: boolean;+ }) {+ const traceContent = OpenlitConfig.traceContent;+ // Non-streaming: result is GenerateContentResult = { response: GenerateContentResponse }+ // Streaming: result is our custom accumulated plain object+ const responseData = result.response || result;+ const config = args[0]?.config || args[1] || {};+ const {+ temperature,+ maxOutputTokens,+ topP,+ topK,+ stopSequences,+ } = config;++ // Request Params attributes+ if (temperature !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);+ }+ if (maxOutputTokens !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, maxOutputTokens);+ }+ if (topP !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, topP);+ }+ if (topK !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_K, topK);+ }+ if (stopSequences) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, stopSequences);+ }+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, isStream);++ if (traceContent) {+ const contents = args[0]?.contents || args[0];+ let messages: any[] = [];+ if (typeof contents === 'string') {+ messages = [{ role: 'user', content: contents }];+ } else if (Array.isArray(contents)) {+ messages = contents.map((item: any) => ({+ role: item.role || 'user',+ content: Array.isArray(item.parts)+ ? item.parts.map((p: any) => p.text || '').join(' ')+ : (item.parts || ''),+ }));+ }+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages));+ }++ span.setAttribute(+ SemanticConvention.GEN_AI_OPERATION,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT+ );++ const model = responseData.modelVersion || responseData.model || args[0]?.model || 'gemini-pro';+ const responseModel = model;++ const pricingInfo = await OpenlitConfig.updatePricingJson(OpenlitConfig.pricing_json);++ // Calculate cost of the operation+ const usageMetadata = responseData.usageMetadata;+ const promptTokens = usageMetadata?.promptTokenCount || 0;+ const completionTokens = usageMetadata?.candidatesTokenCount || 0;++ const cost = OpenLitHelper.getChatModelCost(+ model,+ pricingInfo,+ promptTokens,+ completionTokens+ );++ GoogleAIWrapper.setBaseSpanAttributes(span, {+ genAIEndpoint,+ model,+ user: undefined,+ cost,+ aiSystem: GoogleAIWrapper.aiSystem,+ });++ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, responseModel);++ // Token usage+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, promptTokens);+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, completionTokens);+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, usageMetadata?.totalTokenCount || 0);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, usageMetadata?.totalTokenCount || 0);++ // TTFT and TBT metrics+ if (ttft > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft);+ }+ if (tbt > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt);+ }++ // Finish reason+ if (responseData.candidates && responseData.candidates[0]?.finishReason) {+ span.setAttribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [responseData.candidates[0].finishReason]+ );+ }++ // Output type+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, SemanticConvention.GEN_AI_OUTPUT_TYPE_TEXT);++ // Content+ if (traceContent) {+ const completionContent = (typeof responseData.text === 'function' ? responseData.text() : responseData.text) ||+ (responseData.candidates?.[0]?.content?.parts?.[0]?.text) || '';+ const finishReason = responseData.candidates?.[0]?.finishReason || 'stop';+ span.setAttribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(completionContent, finishReason)+ );+ }++ return {+ genAIEndpoint,+ model,+ user: undefined,+ cost,+ aiSystem: GoogleAIWrapper.aiSystem,+ };+ }+}++export default GoogleAIWrapper;diff --git a/sdk/typescript/src/instrumentation/groq/index.ts b/sdk/typescript/src/instrumentation/groq/index.tsnew file mode 100644index 00000000..8fbd18c1--- /dev/null+++ b/sdk/typescript/src/instrumentation/groq/index.ts@@ -0,0 +1,58 @@+import {+ InstrumentationBase,+ InstrumentationModuleDefinition,+ InstrumentationNodeModuleDefinition,+ isWrapped,+} from '@opentelemetry/instrumentation';+import { InstrumentationConfig } from '@opentelemetry/instrumentation';+import { INSTRUMENTATION_PREFIX } from '../../constant';+import GroqWrapper from './wrapper';++export interface GroqInstrumentationConfig extends InstrumentationConfig {}++export default class OpenlitGroqInstrumentation extends InstrumentationBase {+ constructor(config: GroqInstrumentationConfig = {}) {+ super(`${INSTRUMENTATION_PREFIX}/instrumentation-groq`, '1.0.0', config);+ }++ protected init(): void | InstrumentationModuleDefinition | InstrumentationModuleDefinition[] {+ const module = new InstrumentationNodeModuleDefinition(+ 'groq-sdk',+ ['>=0.5.0'],+ (moduleExports) => {+ this._patch(moduleExports);+ return moduleExports;+ },+ (moduleExports) => {+ if (moduleExports !== undefined) {+ this._unpatch(moduleExports);+ }+ }+ );+ return [module];+ }++ public manualPatch(groq: any): void {+ this._patch(groq);+ }++ protected _patch(moduleExports: any) {+ try {+ if (isWrapped(moduleExports.Groq.Chat.Completions.prototype.create)) {+ this._unwrap(moduleExports.Groq.Chat.Completions.prototype, 'create');+ }++ this._wrap(+ moduleExports.Groq.Chat.Completions.prototype,+ 'create',+ GroqWrapper._patchChatCompletionCreate(this.tracer)+ );+ } catch (e) {+ console.error('Error in _patch method:', e);+ }+ }++ protected _unpatch(moduleExports: any) {+ this._unwrap(moduleExports.Groq.Chat.Completions.prototype, 'create');+ }+}diff --git a/sdk/typescript/src/instrumentation/groq/wrapper.ts b/sdk/typescript/src/instrumentation/groq/wrapper.tsnew file mode 100644index 00000000..8437d64e--- /dev/null+++ b/sdk/typescript/src/instrumentation/groq/wrapper.ts@@ -0,0 +1,392 @@+import { Span, SpanKind, Tracer, context, trace } from '@opentelemetry/api';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import SemanticConvention from '../../semantic-convention';+import BaseWrapper from '../base-wrapper';++class GroqWrapper extends BaseWrapper {+ static aiSystem = 'groq';++ static _patchChatCompletionCreate(tracer: Tracer): any {+ const genAIEndpoint = 'groq.chat.completions';+ return (originalMethod: (...args: any[]) => any) => {+ return async function (this: any, ...args: any[]) {+ const span = tracer.startSpan(genAIEndpoint, { kind: SpanKind.CLIENT });+ return context+ .with(trace.setSpan(context.active(), span), async () => {+ return originalMethod.apply(this, args);+ })+ .then((response: any) => {+ const { stream = false } = args[0];++ if (stream) {+ return OpenLitHelper.createStreamProxy(+ response,+ GroqWrapper._chatCompletionGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ })+ );+ }++ return GroqWrapper._chatCompletion({ args, genAIEndpoint, response, span });+ })+ .catch((e: any) => {+ OpenLitHelper.handleException(span, e);+ span.end();+ throw e;+ });+ };+ };+ }++ static async _chatCompletion({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): Promise<any> {+ let metricParams;+ try {+ metricParams = await GroqWrapper._chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result: response,+ span,+ });+ return response;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ // Record metrics after span has ended if parameters are available+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async *_chatCompletionGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): AsyncGenerator<unknown, any, unknown> {+ let metricParams;+ const timestamps: number[] = [];+ const startTime = Date.now();++ try {+ const { messages } = args[0];+ let { tools } = args[0];+ const result = {+ id: '0',+ created: -1,+ model: '',+ system_fingerprint: '',+ choices: [+ {+ index: 0,+ logprobs: null,+ finish_reason: 'stop',+ message: { role: 'assistant', content: '' },+ },+ ],+ usage: {+ prompt_tokens: 0,+ completion_tokens: 0,+ total_tokens: 0,+ },+ };++ let toolCalls: any[] = [];++ for await (const chunk of response) {+ timestamps.push(Date.now());++ result.id = chunk.id;+ result.created = chunk.created;+ result.model = chunk.model;++ if (chunk.system_fingerprint) {+ result.system_fingerprint = chunk.system_fingerprint;+ }++ if (chunk.choices[0]?.finish_reason) {+ result.choices[0].finish_reason = chunk.choices[0].finish_reason;+ }+ if (chunk.choices[0]?.logprobs) {+ result.choices[0].logprobs = chunk.choices[0].logprobs;+ }+ if (chunk.choices[0]?.delta.content) {+ result.choices[0].message.content += chunk.choices[0].delta.content;+ }++ // Handle tool calls for streaming+ if (chunk.choices[0]?.delta.tool_calls) {+ const deltaTools = chunk.choices[0].delta.tool_calls;++ for (const tool of deltaTools) {+ const idx = tool.index || 0;++ // Extend array if needed+ while (toolCalls.length <= idx) {+ toolCalls.push({+ id: '',+ type: 'function',+ function: { name: '', arguments: '' }+ });+ }++ if (tool.id) {+ // New tool call+ toolCalls[idx].id = tool.id;+ toolCalls[idx].type = tool.type || 'function';+ if (tool.function?.name) {+ toolCalls[idx].function.name = tool.function.name;+ }+ if (tool.function?.arguments) {+ toolCalls[idx].function.arguments = tool.function.arguments;+ }+ } else if (tool.function?.arguments) {+ // Append arguments to existing tool call+ toolCalls[idx].function.arguments += tool.function.arguments;+ }+ }++ tools = true;+ }++ yield chunk;+ }++ if (toolCalls.length > 0) {+ result.choices[0].message = {+ ...result.choices[0].message,+ tool_calls: toolCalls+ } as any;+ }++ // Estimate token usage if not provided+ let promptTokens = 0;+ for (const message of messages || []) {+ promptTokens += OpenLitHelper.openaiTokens(message.content as string, result.model) ?? 0;+ }++ const completionTokens = OpenLitHelper.openaiTokens(+ result.choices[0].message.content ?? '',+ result.model+ );++ if (completionTokens) {+ result.usage = {+ prompt_tokens: promptTokens,+ completion_tokens: completionTokens,+ total_tokens: promptTokens + completionTokens,+ };+ }++ args[0].tools = tools;++ // Calculate TTFT and TBT+ const ttft = timestamps.length > 0 ? (timestamps[0] - startTime) / 1000 : 0;+ let tbt = 0;+ if (timestamps.length > 1) {+ const timeDiffs = timestamps.slice(1).map((t, i) => t - timestamps[i]);+ tbt = timeDiffs.reduce((a, b) => a + b, 0) / timeDiffs.length / 1000;+ }++ metricParams = await GroqWrapper._chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft,+ tbt,+ });++ return result;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ // Record metrics after span has ended if parameters are available+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async _chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft = 0,+ tbt = 0,+ }: {+ args: any[];+ genAIEndpoint: string;+ result: any;+ span: Span;+ ttft?: number;+ tbt?: number;+ }) {+ const traceContent = OpenlitConfig.traceContent;+ const {+ messages,+ frequency_penalty = 0,+ max_tokens = null,+ presence_penalty = 0,+ seed = null,+ stop = null,+ temperature = 1,+ top_p,+ user,+ stream = false,+ } = args[0];++ // Request Params attributes : Start+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, top_p || 1);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, max_tokens || -1);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY, presence_penalty);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, frequency_penalty);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_SEED, seed ? String(seed) : '');+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);+ if (stop) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, Array.isArray(stop) ? stop : [stop]);+ }+ if (user) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_USER, user);+ }++ if (traceContent) {+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages || []));+ }+ // Request Params attributes : End++ span.setAttribute(+ SemanticConvention.GEN_AI_OPERATION,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT+ );++ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_ID, result.id);++ const model = result.model || 'llama-3.1-8b-instant';+ const responseModel = result.model || model;++ const pricingInfo = await OpenlitConfig.updatePricingJson(OpenlitConfig.pricing_json);++ // Calculate cost of the operation+ const cost = OpenLitHelper.getChatModelCost(+ model,+ pricingInfo,+ result.usage.prompt_tokens,+ result.usage.completion_tokens+ );++ GroqWrapper.setBaseSpanAttributes(span, {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: GroqWrapper.aiSystem,+ });++ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, responseModel);++ // Groq-specific attributes+ if (result.system_fingerprint) {+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_SYSTEM_FINGERPRINT, result.system_fingerprint);+ }++ // Token usage+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, result.usage.prompt_tokens);+ span.setAttribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,+ result.usage.completion_tokens+ );+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, result.usage.total_tokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, result.usage.total_tokens);++ // TTFT and TBT metrics+ if (ttft > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft);+ }+ if (tbt > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt);+ }++ // Finish reason+ if (result.choices[0].finish_reason) {+ span.setAttribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [result.choices[0].finish_reason]+ );+ }++ // Output type+ const outputType = typeof result.choices[0].message.content === 'string'+ ? SemanticConvention.GEN_AI_OUTPUT_TYPE_TEXT+ : SemanticConvention.GEN_AI_OUTPUT_TYPE_JSON;+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, outputType);++ // Tool calls handling+ if (result.choices[0].message.tool_calls) {+ const toolCalls = result.choices[0].message.tool_calls;+ const toolNames = toolCalls.map((t: any) => t.function?.name || '').filter(Boolean);+ const toolIds = toolCalls.map((t: any) => t.id || '').filter(Boolean);+ const toolArgs = toolCalls.map((t: any) => t.function?.arguments || '').filter(Boolean);+ const toolTypes = toolCalls.map((t: any) => t.type || '').filter(Boolean);++ if (toolNames.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_NAME, toolNames.join(', '));+ }+ if (toolIds.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_CALL_ID, toolIds.join(', '));+ }+ if (toolArgs.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_CALL_ARGUMENTS, toolArgs);+ }+ if (toolTypes.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_TYPE, toolTypes.join(', '));+ }+ }++ // Content+ if (traceContent) {+ const completionContent = result.choices[0].message.content || '';+ const toolCalls = result.choices[0].message.tool_calls;+ span.setAttribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(completionContent, result.choices[0].finish_reason || 'stop', toolCalls)+ );+ }++ return {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: GroqWrapper.aiSystem,+ };+ }+}++export default GroqWrapper;diff --git a/sdk/typescript/src/instrumentation/index.ts b/sdk/typescript/src/instrumentation/index.tsindex ed243d0d..e0bcb05d 100644--- a/sdk/typescript/src/instrumentation/index.ts+++ b/sdk/typescript/src/instrumentation/index.ts@@ -5,12 +5,22 @@ import { TracerProvider } from '@opentelemetry/api';import OpenAIInstrumentation from './openai';import AnthropicInstrumentation from './anthropic';import CohereInstrumentation from './cohere';+import GroqInstrumentation from './groq';+import MistralInstrumentation from './mistral';+import GoogleAIInstrumentation from './google-ai';+import TogetherInstrumentation from './together';+import OllamaInstrumentation from './ollama';export default class Instrumentations {static availableInstrumentations: OpenlitInstrumentations = {openai: new OpenAIInstrumentation(),anthropic: new AnthropicInstrumentation(),cohere: new CohereInstrumentation(),+ groq: new GroqInstrumentation(),+ mistral: new MistralInstrumentation(),+ 'google-ai': new GoogleAIInstrumentation(),+ together: new TogetherInstrumentation(),+ ollama: new OllamaInstrumentation(),};static setup(diff --git a/sdk/typescript/src/instrumentation/mistral/index.ts b/sdk/typescript/src/instrumentation/mistral/index.tsnew file mode 100644index 00000000..13bafcd3--- /dev/null+++ b/sdk/typescript/src/instrumentation/mistral/index.ts@@ -0,0 +1,77 @@+import {+ InstrumentationBase,+ InstrumentationModuleDefinition,+ InstrumentationNodeModuleDefinition,+ isWrapped,+} from '@opentelemetry/instrumentation';+import { InstrumentationConfig } from '@opentelemetry/instrumentation';+import { INSTRUMENTATION_PREFIX } from '../../constant';+import MistralWrapper from './wrapper';++export interface MistralInstrumentationConfig extends InstrumentationConfig {}++export default class OpenlitMistralInstrumentation extends InstrumentationBase {+ constructor(config: MistralInstrumentationConfig = {}) {+ super(`${INSTRUMENTATION_PREFIX}/instrumentation-mistral`, '1.0.0', config);+ }++ protected init(): void | InstrumentationModuleDefinition | InstrumentationModuleDefinition[] {+ const module = new InstrumentationNodeModuleDefinition(+ '@mistralai/mistralai',+ ['>=1.0.0'],+ (moduleExports) => {+ this._patch(moduleExports);+ return moduleExports;+ },+ (moduleExports) => {+ if (moduleExports !== undefined) {+ this._unpatch(moduleExports);+ }+ }+ );+ return [module];+ }++ public manualPatch(mistral: any): void {+ this._patch(mistral);+ }++ protected _patch(moduleExports: any) {+ try {+ // Chat and Embeddings are instance properties in the new SDK β get their+ // prototypes via a dummy instance (no API calls are made at construction time)+ const dummy = new moduleExports.Mistral({ apiKey: 'dummy' });+ const ChatProto = Object.getPrototypeOf(dummy.chat);+ const EmbeddingsProto = Object.getPrototypeOf(dummy.embeddings);++ if (isWrapped(ChatProto.complete)) {+ this._unwrap(ChatProto, 'complete');+ }+ if (isWrapped(ChatProto.stream)) {+ this._unwrap(ChatProto, 'stream');+ }+ if (isWrapped(EmbeddingsProto.create)) {+ this._unwrap(EmbeddingsProto, 'create');+ }++ this._wrap(ChatProto, 'complete', MistralWrapper._patchChatCompletionCreate(this.tracer));+ this._wrap(ChatProto, 'stream', MistralWrapper._patchChatCompletionCreate(this.tracer));+ this._wrap(EmbeddingsProto, 'create', MistralWrapper._patchEmbedding(this.tracer));+ } catch (e) {+ console.error('Error in _patch method:', e);+ }+ }++ protected _unpatch(moduleExports: any) {+ try {+ const dummy = new moduleExports.Mistral({ apiKey: 'dummy' });+ const ChatProto = Object.getPrototypeOf(dummy.chat);+ const EmbeddingsProto = Object.getPrototypeOf(dummy.embeddings);+ this._unwrap(ChatProto, 'complete');+ this._unwrap(ChatProto, 'stream');+ this._unwrap(EmbeddingsProto, 'create');+ } catch (e) {+ console.error('Error in _unpatch method:', e);+ }+ }+}diff --git a/sdk/typescript/src/instrumentation/mistral/wrapper.ts b/sdk/typescript/src/instrumentation/mistral/wrapper.tsnew file mode 100644index 00000000..35524e4b--- /dev/null+++ b/sdk/typescript/src/instrumentation/mistral/wrapper.ts@@ -0,0 +1,445 @@+import { Span, SpanKind, Tracer, context, trace } from '@opentelemetry/api';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import SemanticConvention from '../../semantic-convention';+import BaseWrapper, { BaseSpanAttributes } from '../base-wrapper';++class MistralWrapper extends BaseWrapper {+ static aiSystem = 'mistral';++ static _patchChatCompletionCreate(tracer: Tracer): any {+ const genAIEndpoint = 'mistral.chat.completions';+ return (originalMethod: (...args: any[]) => any) => {+ return async function (this: any, ...args: any[]) {+ const span = tracer.startSpan(genAIEndpoint, { kind: SpanKind.CLIENT });+ return context+ .with(trace.setSpan(context.active(), span), async () => {+ return originalMethod.apply(this, args);+ })+ .then((response: any) => {+ // Detect streaming: new Mistral SDK's chat.stream() returns an async iterable directly+ const isStream = args[0]?.stream === true || typeof response[Symbol.asyncIterator] === 'function';++ if (isStream) {+ return OpenLitHelper.createStreamProxy(+ response,+ MistralWrapper._chatCompletionGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ })+ );+ }++ return MistralWrapper._chatCompletion({ args, genAIEndpoint, response, span });+ })+ .catch((e: any) => {+ OpenLitHelper.handleException(span, e);+ span.end();+ throw e;+ });+ };+ };+ }++ static async _chatCompletion({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): Promise<any> {+ let metricParams;+ try {+ metricParams = await MistralWrapper._chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result: response,+ span,+ });+ return response;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async *_chatCompletionGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): AsyncGenerator<unknown, any, unknown> {+ let metricParams;+ const timestamps: number[] = [];+ const startTime = Date.now();++ try {+ const { messages } = args[0];+ let { tools } = args[0];+ const result = {+ id: '0',+ created: -1,+ model: '',+ choices: [+ {+ index: 0,+ finish_reason: 'stop',+ message: { role: 'assistant', content: '' },+ },+ ],+ usage: {+ prompt_tokens: 0,+ completion_tokens: 0,+ total_tokens: 0,+ },+ };++ let toolCalls: any[] = [];++ for await (const chunk of response) {+ timestamps.push(Date.now());++ // New Mistral SDK wraps each SSE event in { data: { ... } }+ const chunkData = chunk.data ?? chunk;++ result.id = chunkData.id || result.id;+ result.created = chunkData.created || result.created;+ result.model = chunkData.model || result.model;++ if (chunkData.choices && chunkData.choices[0]) {+ if (chunkData.choices[0].finish_reason) {+ result.choices[0].finish_reason = chunkData.choices[0].finish_reason;+ }+ if (chunkData.choices[0].delta?.content) {+ result.choices[0].message.content += chunkData.choices[0].delta.content;+ }++ // Handle tool calls for streaming+ if (chunkData.choices[0].delta?.tool_calls) {+ const deltaTools = chunkData.choices[0].delta.tool_calls;++ for (const tool of deltaTools) {+ const idx = tool.index || 0;++ while (toolCalls.length <= idx) {+ toolCalls.push({+ id: '',+ type: 'function',+ function: { name: '', arguments: '' }+ });+ }++ if (tool.id) {+ toolCalls[idx].id = tool.id;+ toolCalls[idx].type = tool.type || 'function';+ if (tool.function?.name) {+ toolCalls[idx].function.name = tool.function.name;+ }+ if (tool.function?.arguments) {+ toolCalls[idx].function.arguments = tool.function.arguments;+ }+ } else if (tool.function?.arguments) {+ toolCalls[idx].function.arguments += tool.function.arguments;+ }+ }++ tools = true;+ }+ }++ yield chunk;+ }++ if (toolCalls.length > 0) {+ result.choices[0].message = {+ ...result.choices[0].message,+ tool_calls: toolCalls+ } as any;+ }++ // Estimate token usage if not provided+ let promptTokens = 0;+ for (const message of messages || []) {+ promptTokens += OpenLitHelper.openaiTokens(message.content as string, result.model) ?? 0;+ }++ const completionTokens = OpenLitHelper.openaiTokens(+ result.choices[0].message.content ?? '',+ result.model+ );++ if (completionTokens) {+ result.usage = {+ prompt_tokens: promptTokens,+ completion_tokens: completionTokens,+ total_tokens: promptTokens + completionTokens,+ };+ }++ args[0].tools = tools;++ // Calculate TTFT and TBT+ const ttft = timestamps.length > 0 ? (timestamps[0] - startTime) / 1000 : 0;+ let tbt = 0;+ if (timestamps.length > 1) {+ const timeDiffs = timestamps.slice(1).map((t, i) => t - timestamps[i]);+ tbt = timeDiffs.reduce((a, b) => a + b, 0) / timeDiffs.length / 1000;+ }++ metricParams = await MistralWrapper._chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft,+ tbt,+ });++ return result;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async _chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft = 0,+ tbt = 0,+ }: {+ args: any[];+ genAIEndpoint: string;+ result: any;+ span: Span;+ ttft?: number;+ tbt?: number;+ }) {+ const traceContent = OpenlitConfig.traceContent;+ const {+ messages,+ max_tokens = null,+ temperature = 0.7,+ top_p,+ user,+ stream = false,+ safe_prompt = false,+ } = args[0];++ // Request Params attributes+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, top_p || 1);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, max_tokens || -1);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);++ if (user) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_USER, user);+ }+ if (safe_prompt !== undefined) {+ span.setAttribute('gen_ai.request.safe_prompt', safe_prompt);+ }++ if (traceContent) {+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages || []));+ }++ span.setAttribute(+ SemanticConvention.GEN_AI_OPERATION,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT+ );++ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_ID, result.id);++ const model = result.model || 'mistral-small-latest';+ const responseModel = result.model || model;++ const pricingInfo = await OpenlitConfig.updatePricingJson(OpenlitConfig.pricing_json);++ // Support both camelCase (new SDK) and snake_case (old SDK) usage fields+ const promptTokens = result.usage?.promptTokens ?? result.usage?.prompt_tokens ?? 0;+ const completionTokens = result.usage?.completionTokens ?? result.usage?.completion_tokens ?? 0;+ const totalTokens = result.usage?.totalTokens ?? result.usage?.total_tokens ?? 0;++ // Calculate cost of the operation+ const cost = OpenLitHelper.getChatModelCost(model, pricingInfo, promptTokens, completionTokens);++ MistralWrapper.setBaseSpanAttributes(span, {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: MistralWrapper.aiSystem,+ });++ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, responseModel);++ // Token usage+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, promptTokens);+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, completionTokens);+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, totalTokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, totalTokens);++ // TTFT and TBT metrics+ if (ttft > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft);+ }+ if (tbt > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt);+ }++ // Finish reason+ if (result.choices[0].finish_reason) {+ span.setAttribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [result.choices[0].finish_reason]+ );+ }++ // Output type+ const outputType = typeof result.choices[0].message.content === 'string'+ ? SemanticConvention.GEN_AI_OUTPUT_TYPE_TEXT+ : SemanticConvention.GEN_AI_OUTPUT_TYPE_JSON;+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, outputType);++ // Tool calls handling+ if (result.choices[0].message.tool_calls) {+ const toolCalls = result.choices[0].message.tool_calls;+ const toolNames = toolCalls.map((t: any) => t.function?.name || '').filter(Boolean);+ const toolIds = toolCalls.map((t: any) => t.id || '').filter(Boolean);+ const toolArgs = toolCalls.map((t: any) => t.function?.arguments || '').filter(Boolean);+ const toolTypes = toolCalls.map((t: any) => t.type || '').filter(Boolean);++ if (toolNames.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_NAME, toolNames.join(', '));+ }+ if (toolIds.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_CALL_ID, toolIds.join(', '));+ }+ if (toolArgs.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_CALL_ARGUMENTS, toolArgs);+ }+ if (toolTypes.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_TYPE, toolTypes.join(', '));+ }+ }++ // Content+ if (traceContent) {+ const completionContent = result.choices[0].message.content || '';+ const toolCalls = result.choices[0].message.tool_calls;+ span.setAttribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(completionContent, result.choices[0].finish_reason || 'stop', toolCalls)+ );+ }++ return {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: MistralWrapper.aiSystem,+ };+ }++ static _patchEmbedding(tracer: Tracer): any {+ const genAIEndpoint = 'mistral.embeddings';+ const traceContent = OpenlitConfig.traceContent;++ return (originalMethod: (...args: any[]) => any) => {+ return async function (this: any, ...args: any[]) {+ const span = tracer.startSpan(genAIEndpoint, { kind: SpanKind.CLIENT });+ return context.with(trace.setSpan(context.active(), span), async () => {+ let metricParams: BaseSpanAttributes | undefined;+ try {+ const response = await originalMethod.apply(this, args);++ const model = args[0].model || 'mistral-embed';+ const pricingInfo = await OpenlitConfig.updatePricingJson(OpenlitConfig.pricing_json);+ // Mistral SDK returns camelCase usage fields+ const promptTokens = response.usage?.promptTokens ?? response.usage?.prompt_tokens ?? 0;+ const totalTokens = response.usage?.totalTokens ?? response.usage?.total_tokens ?? 0;+ const cost = OpenLitHelper.getEmbedModelCost(model, pricingInfo, promptTokens);++ span.setAttribute(+ SemanticConvention.GEN_AI_OPERATION,+ SemanticConvention.GEN_AI_OPERATION_TYPE_EMBEDDING+ );++ const { input, inputs, user, encoding_format = 'float' } = args[0];+ const embeddingInput = input ?? inputs;++ MistralWrapper.setBaseSpanAttributes(span, {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: MistralWrapper.aiSystem,+ });++ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_ENCODING_FORMATS, [encoding_format]);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, false);+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TBT, 0);+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TTFT, 0);++ if (user) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_USER, user);+ }+ if (traceContent && embeddingInput) {+ const inputArr = Array.isArray(embeddingInput) ? embeddingInput : [embeddingInput];+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(inputArr.map((c: string) => ({ role: 'user', content: c }))));+ }++ span.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, promptTokens);+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, totalTokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, promptTokens);++ metricParams = {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: MistralWrapper.aiSystem,+ };++ return response;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ });+ };+ };+ }+}++export default MistralWrapper;diff --git a/sdk/typescript/src/instrumentation/ollama/wrapper.ts b/sdk/typescript/src/instrumentation/ollama/wrapper.tsindex 82ca1fcd..679c7faa 100644--- a/sdk/typescript/src/instrumentation/ollama/wrapper.ts+++ b/sdk/typescript/src/instrumentation/ollama/wrapper.ts@@ -1,30 +1,11 @@-import { Span, SpanKind, SpanStatusCode, Tracer, context, trace } from '@opentelemetry/api';+import { Span, SpanKind, Tracer, context, trace } from '@opentelemetry/api';import OpenlitConfig from '../../config';import OpenLitHelper from '../../helpers';import SemanticConvention from '../../semantic-convention';-import { SDK_NAME, TELEMETRY_SDK_NAME } from '../../constant';-import BaseWrapper from '../base-wrapper';+import BaseWrapper, { BaseSpanAttributes } from '../base-wrapper';export default class OllamaWrapper extends BaseWrapper {- static setBaseSpanAttributes(- span: any,- { genAIEndpoint, model, user, cost, environment, applicationName }: any- ) {- span.setAttributes({- [TELEMETRY_SDK_NAME]: SDK_NAME,- });-- span.setAttribute(TELEMETRY_SDK_NAME, SDK_NAME);- span.setAttribute(SemanticConvention.GEN_AI_PROVIDER_NAME, SemanticConvention.GEN_AI_SYSTEM_ANTHROPIC);- span.setAttribute(SemanticConvention.GEN_AI_ENDPOINT, genAIEndpoint);- span.setAttribute(SemanticConvention.GEN_AI_ENVIRONMENT, environment);- span.setAttribute(SemanticConvention.GEN_AI_APPLICATION_NAME, applicationName);- span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MODEL, model);- span.setAttribute(SemanticConvention.GEN_AI_REQUEST_USER, user);- if (cost !== undefined) span.setAttribute(SemanticConvention.GEN_AI_USAGE_COST, cost);-- span.setStatus({ code: SpanStatusCode.OK });- }+ static aiSystem = 'ollama';static _patchChat(tracer: Tracer): any {const genAIEndpoint = 'ollama.chat';@@ -71,8 +52,9 @@ export default class OllamaWrapper extends BaseWrapper {response: any;span: Span;}) {+ let metricParams: BaseSpanAttributes | undefined;try {- await OllamaWrapper._chatCommonSetter({+ metricParams = await OllamaWrapper._chatCommonSetter({args,genAIEndpoint,result: response,@@ -83,6 +65,9 @@ export default class OllamaWrapper extends BaseWrapper {OpenLitHelper.handleException(span, e);} finally {span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }}}@@ -97,70 +82,57 @@ export default class OllamaWrapper extends BaseWrapper {response: any;span: Span;}) {+ let metricParams: BaseSpanAttributes | undefined;+ const timestamps: number[] = [];+ const startTime = Date.now();try {- const result = {- id: '0',+ const result: any = {model: '',- stop_reason: '',- content: [- {- text: '',- role: '',- },- ],- usage: {- input_tokens: 0,- output_tokens: 0,- total_tokens: 0,- },+ message: { role: 'assistant', content: '' },+ done_reason: '',+ prompt_eval_count: 0,+ eval_count: 0,};- for await (const chunk of response) {- switch (chunk.type) {- case 'content_block_delta':- result.content[0].text += chunk.delta?.text ?? '';- break;- case 'message_stop':- break;- case 'content_block_stop':- break;-- case 'message_start':- if (chunk.message) {- result.id = chunk.message.id;- result.model = chunk.message.model;- result.content[0].role = chunk.message.role;- result.usage.input_tokens += Number(chunk.message.usage?.input_tokens) ?? 0;- result.usage.output_tokens += Number(chunk.message.usage?.output_tokens) ?? 0;- result.stop_reason = chunk.message?.stop_reason ?? '';- }- break;-- case 'content_block_start':- result.content[0].text = chunk.content_block?.text ?? '';- break;- case 'message_delta':- result.stop_reason = chunk.delta?.stop_reason ?? '';- result.usage.output_tokens += Number(chunk.usage?.output_tokens) ?? 0;- break;+ for await (const chunk of response) {+ timestamps.push(Date.now());+ result.model = chunk.model || result.model;+ if (chunk.message?.content) {+ result.message.content += chunk.message.content;+ result.message.role = chunk.message.role || result.message.role;+ }+ if (chunk.done) {+ result.done_reason = chunk.done_reason || '';+ result.prompt_eval_count = chunk.prompt_eval_count || 0;+ result.eval_count = chunk.eval_count || 0;}yield chunk;}- result.usage.total_tokens = result.usage.output_tokens + result.usage.input_tokens;+ const ttft = timestamps.length > 0 ? (timestamps[0] - startTime) / 1000 : 0;+ let tbt = 0;+ if (timestamps.length > 1) {+ const timeDiffs = timestamps.slice(1).map((t, i) => t - timestamps[i]);+ tbt = timeDiffs.reduce((a, b) => a + b, 0) / timeDiffs.length / 1000;+ }- await OllamaWrapper._chatCommonSetter({+ metricParams = await OllamaWrapper._chatCommonSetter({args,genAIEndpoint,result,span,+ ttft,+ tbt,});return response;} catch (e: any) {OpenLitHelper.handleException(span, e);} finally {span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }}}@@ -169,14 +141,16 @@ export default class OllamaWrapper extends BaseWrapper {genAIEndpoint,result,span,+ ttft = 0,+ tbt = 0,}: {args: any[];genAIEndpoint: string;result: any;span: Span;+ ttft?: number;+ tbt?: number;}) {- const applicationName = OpenlitConfig.applicationName;- const environment = OpenlitConfig.environment;const traceContent = OpenlitConfig.traceContent;const {messages,@@ -187,34 +161,8 @@ export default class OllamaWrapper extends BaseWrapper {top_k,user,stream = false,- stop_reason,} = args[0];- // Format 'messages' into a single string- const messagePrompt = messages || '';- const formattedMessages = [];-- for (const message of messagePrompt) {- const role = message.role;- const content = message.content;-- if (Array.isArray(content)) {- const contentStr = content- .map((item) => {- if ('type' in item) {- return `${item.type}: ${item.text ? item.text : item.image_url}`;- } else {- return `text: ${item.text}`;- }- })- .join(', ');- formattedMessages.push(`${role}: ${contentStr}`);- } else {- formattedMessages.push(`${role}: ${content}`);- }- }-- const prompt = formattedMessages.join('\n');span.setAttribute(SemanticConvention.GEN_AI_OPERATION, SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT);span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_ID, result.id);@@ -222,8 +170,8 @@ export default class OllamaWrapper extends BaseWrapper {const pricingInfo = await OpenlitConfig.updatePricingJson(OpenlitConfig.pricing_json);- const promptTokens = result.prompt_eval_count;- const completionTokens = result.eval_count;+ const promptTokens = result.prompt_eval_count || 0;+ const completionTokens = result.eval_count || 0;const totalTokens = promptTokens + completionTokens;// Calculate cost of the operation@@ -234,37 +182,54 @@ export default class OllamaWrapper extends BaseWrapper {model,user,cost,- applicationName,- environment,+ aiSystem: OllamaWrapper.aiSystem,});+ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, model);+// Request Params attributes : Startspan.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, top_p);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_K, top_k);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, max_tokens);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);- span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, stop_reason);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_SEED, seed);if (traceContent) {- span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt);+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages || []));}// Request Params attributes : Endspan.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, promptTokens);span.setAttribute(SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS, completionTokens);span.setAttribute(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, totalTokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, totalTokens);if (result.done_reason) {- span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, result.done_reason);+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON, [result.done_reason]);+ }++ // TTFT and TBT metrics+ if (ttft > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft);+ }+ if (tbt > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt);}if (traceContent) {- // Format 'messages' into a single stringconst { message = {} } = result;- const messageString = `${message.role}: ${message.content}`;- span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, messageString);+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(message.content || '', result.done_reason || 'stop'));}++ return {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: OllamaWrapper.aiSystem,+ };}}diff --git a/sdk/typescript/src/instrumentation/openai/wrapper.ts b/sdk/typescript/src/instrumentation/openai/wrapper.tsindex d4943710..e7b69f9c 100644--- a/sdk/typescript/src/instrumentation/openai/wrapper.ts+++ b/sdk/typescript/src/instrumentation/openai/wrapper.ts@@ -286,32 +286,7 @@ class OpenAIWrapper extends BaseWrapper {}if (traceContent) {- // Format 'messages' into a single string- const messagePrompt = messages || [];- const formattedMessages = [];-- for (const message of messagePrompt) {- const role = message.role;- const content = message.content;-- if (Array.isArray(content)) {- const contentStr = content- .map((item) => {- if ('type' in item) {- return `${item.type}: ${item.text ? item.text : item.image_url}`;- } else {- return `text: ${item.text}`;- }- })- .join(', ');- formattedMessages.push(`${role}: ${contentStr}`);- } else {- formattedMessages.push(`${role}: ${content}`);- }- }-- const prompt = formattedMessages.join('\n');- span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt);+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages || []));}// Request Params attributes : End@@ -444,27 +419,13 @@ class OpenAIWrapper extends BaseWrapper {// Contentif (traceContent) {- // Format completion content - use actual content or empty string if only tool calls- const completionContent = result.choices[0].message.content || '';-- if (n === 1) {- span.setAttribute(- SemanticConvention.GEN_AI_OUTPUT_MESSAGES,- completionContent- );- } else {- let i = 0;- while (i < n) {- const attribute_name = `${SemanticConvention.GEN_AI_OUTPUT_MESSAGES}.${i}`;- span.setAttribute(attribute_name, result.choices[i].message.content || '');- i += 1;- }- }-- // Add events for backward compatibility- span.addEvent(SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT, {- [SemanticConvention.GEN_AI_OUTPUT_MESSAGES]: completionContent,- });+ const toolCalls = result.choices[0].message.tool_calls;+ const outputJson = OpenLitHelper.buildOutputMessages(+ result.choices[0].message.content || '',+ result.choices[0].finish_reason || 'stop',+ toolCalls+ );+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, outputJson);}return {@@ -1109,48 +1070,23 @@ class OpenAIWrapper extends BaseWrapper {stream = false,} = args[0];- // Format input for prompt- let prompt = '';- if (typeof input === 'string') {- prompt = input;- } else if (Array.isArray(input)) {- const formattedMessages = [];- for (const item of input) {- const role = item.role || 'user';- const content = item.content;-- if (typeof content === 'string') {- formattedMessages.push(`${role}: ${content}`);- } else if (Array.isArray(content)) {- const contentParts = content- .map((part: any) => {- if (part.type === 'input_text') {- return `text: ${part.text || ''}`;- } else if (part.type === 'input_image' && part.image_url && !part.image_url.startsWith('data:')) {- return `image_url: ${part.image_url}`;- }- return '';- })- .filter(Boolean)- .join(', ');- formattedMessages.push(`${role}: ${contentParts}`);- }- }- prompt = formattedMessages.join('\n');- }+ // Normalize Responses API input to messages array for buildInputMessages+ const responsesMessages = typeof input === 'string'+ ? [{ role: 'user', content: input }]+ : (Array.isArray(input) ? input : []);// Request Params attributesspan.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, top_p);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, max_output_tokens || -1);span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);-+if (reasoning?.effort) {span.setAttribute('gen_ai.request.reasoning_effort', reasoning.effort);}if (traceContent) {- span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, prompt);+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(responsesMessages));}span.setAttribute(@@ -1251,13 +1187,8 @@ class OpenAIWrapper extends BaseWrapper {// Contentif (traceContent) {- // Set completion content - use actual text or empty string if only tool calls- span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES, completionText);-- // Add events for backward compatibility- span.addEvent(SemanticConvention.GEN_AI_CONTENT_COMPLETION_EVENT, {- [SemanticConvention.GEN_AI_OUTPUT_MESSAGES]: completionText,- });+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(completionText, result.status || 'stop'));}return {diff --git a/sdk/typescript/src/instrumentation/together/index.ts b/sdk/typescript/src/instrumentation/together/index.tsnew file mode 100644index 00000000..84a051b1--- /dev/null+++ b/sdk/typescript/src/instrumentation/together/index.ts@@ -0,0 +1,58 @@+import {+ InstrumentationBase,+ InstrumentationModuleDefinition,+ InstrumentationNodeModuleDefinition,+ isWrapped,+} from '@opentelemetry/instrumentation';+import { InstrumentationConfig } from '@opentelemetry/instrumentation';+import { INSTRUMENTATION_PREFIX } from '../../constant';+import TogetherWrapper from './wrapper';++export interface TogetherInstrumentationConfig extends InstrumentationConfig {}++export default class OpenlitTogetherInstrumentation extends InstrumentationBase {+ constructor(config: TogetherInstrumentationConfig = {}) {+ super(`${INSTRUMENTATION_PREFIX}/instrumentation-together`, '1.0.0', config);+ }++ protected init(): void | InstrumentationModuleDefinition | InstrumentationModuleDefinition[] {+ const module = new InstrumentationNodeModuleDefinition(+ 'together-ai',+ ['>=0.1.0'],+ (moduleExports) => {+ this._patch(moduleExports);+ return moduleExports;+ },+ (moduleExports) => {+ if (moduleExports !== undefined) {+ this._unpatch(moduleExports);+ }+ }+ );+ return [module];+ }++ public manualPatch(together: any): void {+ this._patch(together);+ }++ protected _patch(moduleExports: any) {+ try {+ if (isWrapped(moduleExports.Together.Chat.Completions.prototype.create)) {+ this._unwrap(moduleExports.Together.Chat.Completions.prototype, 'create');+ }++ this._wrap(+ moduleExports.Together.Chat.Completions.prototype,+ 'create',+ TogetherWrapper._patchChatCompletionCreate(this.tracer)+ );+ } catch (e) {+ console.error('Error in _patch method:', e);+ }+ }++ protected _unpatch(moduleExports: any) {+ this._unwrap(moduleExports.Together.Chat.Completions.prototype, 'create');+ }+}diff --git a/sdk/typescript/src/instrumentation/together/wrapper.ts b/sdk/typescript/src/instrumentation/together/wrapper.tsnew file mode 100644index 00000000..a4199c95--- /dev/null+++ b/sdk/typescript/src/instrumentation/together/wrapper.ts@@ -0,0 +1,368 @@+import { Span, SpanKind, Tracer, context, trace } from '@opentelemetry/api';+import OpenlitConfig from '../../config';+import OpenLitHelper from '../../helpers';+import SemanticConvention from '../../semantic-convention';+import BaseWrapper from '../base-wrapper';++class TogetherWrapper extends BaseWrapper {+ static aiSystem = 'together';++ static _patchChatCompletionCreate(tracer: Tracer): any {+ const genAIEndpoint = 'together.chat.completions';+ return (originalMethod: (...args: any[]) => any) => {+ return async function (this: any, ...args: any[]) {+ const span = tracer.startSpan(genAIEndpoint, { kind: SpanKind.CLIENT });+ return context+ .with(trace.setSpan(context.active(), span), async () => {+ return originalMethod.apply(this, args);+ })+ .then((response: any) => {+ const { stream = false } = args[0];++ if (stream) {+ return OpenLitHelper.createStreamProxy(+ response,+ TogetherWrapper._chatCompletionGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ })+ );+ }++ return TogetherWrapper._chatCompletion({ args, genAIEndpoint, response, span });+ })+ .catch((e: any) => {+ OpenLitHelper.handleException(span, e);+ span.end();+ throw e;+ });+ };+ };+ }++ static async _chatCompletion({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): Promise<any> {+ let metricParams;+ try {+ metricParams = await TogetherWrapper._chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result: response,+ span,+ });+ return response;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async *_chatCompletionGenerator({+ args,+ genAIEndpoint,+ response,+ span,+ }: {+ args: any[];+ genAIEndpoint: string;+ response: any;+ span: Span;+ }): AsyncGenerator<unknown, any, unknown> {+ let metricParams;+ const timestamps: number[] = [];+ const startTime = Date.now();++ try {+ const { messages } = args[0];+ const result = {+ id: '0',+ model: args[0].model || '', // model is not in Together streaming chunks+ choices: [+ {+ index: 0,+ finish_reason: 'stop',+ message: { role: 'assistant', content: '' },+ },+ ],+ usage: {+ prompt_tokens: 0,+ completion_tokens: 0,+ total_tokens: 0,+ },+ };++ let toolCalls: any[] = [];++ for await (const chunk of response) {+ timestamps.push(Date.now());++ // id is only present on the first chunk+ if (chunk.id) result.id = chunk.id;++ if (chunk.choices[0]?.finish_reason) {+ result.choices[0].finish_reason = chunk.choices[0].finish_reason;+ }+ if (chunk.choices[0]?.delta?.content) {+ result.choices[0].message.content += chunk.choices[0].delta.content;+ }++ // Handle tool calls for streaming+ if (chunk.choices[0]?.delta?.tool_calls) {+ const deltaTools = chunk.choices[0].delta.tool_calls;++ for (const tool of deltaTools) {+ const idx = tool.index || 0;++ while (toolCalls.length <= idx) {+ toolCalls.push({+ id: '',+ type: 'function',+ function: { name: '', arguments: '' }+ });+ }++ if (tool.id) {+ toolCalls[idx].id = tool.id;+ toolCalls[idx].type = tool.type || 'function';+ if (tool.function?.name) {+ toolCalls[idx].function.name = tool.function.name;+ }+ if (tool.function?.arguments) {+ toolCalls[idx].function.arguments = tool.function.arguments;+ }+ } else if (tool.function?.arguments) {+ toolCalls[idx].function.arguments += tool.function.arguments;+ }+ }++ }++ yield chunk;+ }++ if (toolCalls.length > 0) {+ result.choices[0].message = {+ ...result.choices[0].message,+ tool_calls: toolCalls+ } as any;+ }++ // Fall back to token estimation if usage wasn't returned in the stream+ if (!result.usage.prompt_tokens && !result.usage.completion_tokens) {+ let promptTokens = 0;+ for (const message of messages || []) {+ promptTokens += OpenLitHelper.openaiTokens(message.content as string, result.model) ?? 0;+ }+ const completionTokens = OpenLitHelper.openaiTokens(+ result.choices[0].message.content ?? '',+ result.model+ );+ if (completionTokens) {+ result.usage = {+ prompt_tokens: promptTokens,+ completion_tokens: completionTokens,+ total_tokens: promptTokens + completionTokens,+ };+ }+ }++ // Calculate TTFT and TBT+ const ttft = timestamps.length > 0 ? (timestamps[0] - startTime) / 1000 : 0;+ let tbt = 0;+ if (timestamps.length > 1) {+ const timeDiffs = timestamps.slice(1).map((t, i) => t - timestamps[i]);+ tbt = timeDiffs.reduce((a, b) => a + b, 0) / timeDiffs.length / 1000;+ }++ metricParams = await TogetherWrapper._chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft,+ tbt,+ });++ return result;+ } catch (e: any) {+ OpenLitHelper.handleException(span, e);+ throw e;+ } finally {+ span.end();+ if (metricParams) {+ BaseWrapper.recordMetrics(span, metricParams);+ }+ }+ }++ static async _chatCompletionCommonSetter({+ args,+ genAIEndpoint,+ result,+ span,+ ttft = 0,+ tbt = 0,+ }: {+ args: any[];+ genAIEndpoint: string;+ result: any;+ span: Span;+ ttft?: number;+ tbt?: number;+ }) {+ const traceContent = OpenlitConfig.traceContent;+ const {+ messages,+ frequency_penalty = 0,+ max_tokens = null,+ presence_penalty = 0,+ stop = null,+ temperature = 0.7,+ top_p,+ top_k,+ user,+ stream = false,+ } = args[0];++ // Request Params attributes+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_P, top_p || 1);+ if (top_k !== undefined) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TOP_K, top_k);+ }+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_MAX_TOKENS, max_tokens || -1);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_TEMPERATURE, temperature);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_PRESENCE_PENALTY, presence_penalty);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_FREQUENCY_PENALTY, frequency_penalty);+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_IS_STREAM, stream);+ if (stop) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_STOP_SEQUENCES, Array.isArray(stop) ? stop : [stop]);+ }+ if (user) {+ span.setAttribute(SemanticConvention.GEN_AI_REQUEST_USER, user);+ }++ if (traceContent) {+ span.setAttribute(SemanticConvention.GEN_AI_INPUT_MESSAGES, OpenLitHelper.buildInputMessages(messages || []));+ }++ span.setAttribute(+ SemanticConvention.GEN_AI_OPERATION,+ SemanticConvention.GEN_AI_OPERATION_TYPE_CHAT+ );++ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_ID, result.id);++ const model = result.model || args[0].model || 'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo';+ const responseModel = result.model || model;++ const pricingInfo = await OpenlitConfig.updatePricingJson(OpenlitConfig.pricing_json);++ // Calculate cost of the operation+ const cost = OpenLitHelper.getChatModelCost(+ model,+ pricingInfo,+ result.usage.prompt_tokens,+ result.usage.completion_tokens+ );++ TogetherWrapper.setBaseSpanAttributes(span, {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: TogetherWrapper.aiSystem,+ });++ // Response model+ span.setAttribute(SemanticConvention.GEN_AI_RESPONSE_MODEL, responseModel);++ // Token usage+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_INPUT_TOKENS, result.usage.prompt_tokens);+ span.setAttribute(+ SemanticConvention.GEN_AI_USAGE_OUTPUT_TOKENS,+ result.usage.completion_tokens+ );+ span.setAttribute(SemanticConvention.GEN_AI_USAGE_TOTAL_TOKENS, result.usage.total_tokens);+ span.setAttribute(SemanticConvention.GEN_AI_CLIENT_TOKEN_USAGE, result.usage.total_tokens);++ // TTFT and TBT metrics+ if (ttft > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TTFT, ttft);+ }+ if (tbt > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_SERVER_TBT, tbt);+ }++ // Finish reason+ if (result.choices[0].finish_reason) {+ span.setAttribute(+ SemanticConvention.GEN_AI_RESPONSE_FINISH_REASON,+ [result.choices[0].finish_reason]+ );+ }++ // Output type+ const outputType = typeof result.choices[0].message.content === 'string'+ ? SemanticConvention.GEN_AI_OUTPUT_TYPE_TEXT+ : SemanticConvention.GEN_AI_OUTPUT_TYPE_JSON;+ span.setAttribute(SemanticConvention.GEN_AI_OUTPUT_TYPE, outputType);++ // Tool calls handling+ if (result.choices[0].message.tool_calls) {+ const toolCalls = result.choices[0].message.tool_calls;+ const toolNames = toolCalls.map((t: any) => t.function?.name || '').filter(Boolean);+ const toolIds = toolCalls.map((t: any) => t.id || '').filter(Boolean);+ const toolArgs = toolCalls.map((t: any) => t.function?.arguments || '').filter(Boolean);+ const toolTypes = toolCalls.map((t: any) => t.type || '').filter(Boolean);++ if (toolNames.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_NAME, toolNames.join(', '));+ }+ if (toolIds.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_CALL_ID, toolIds.join(', '));+ }+ if (toolArgs.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_CALL_ARGUMENTS, toolArgs);+ }+ if (toolTypes.length > 0) {+ span.setAttribute(SemanticConvention.GEN_AI_TOOL_TYPE, toolTypes.join(', '));+ }+ }++ // Content+ if (traceContent) {+ const completionContent = result.choices[0].message.content || '';+ const toolCalls = result.choices[0].message.tool_calls;+ span.setAttribute(+ SemanticConvention.GEN_AI_OUTPUT_MESSAGES,+ OpenLitHelper.buildOutputMessages(completionContent, result.choices[0].finish_reason || 'stop', toolCalls)+ );+ }++ return {+ genAIEndpoint,+ model,+ user,+ cost,+ aiSystem: TogetherWrapper.aiSystem,+ };+ }+}++export default TogetherWrapper;diff --git a/sdk/typescript/src/semantic-convention.ts b/sdk/typescript/src/semantic-convention.tsindex 8855cba7..2c82c635 100644--- a/sdk/typescript/src/semantic-convention.ts+++ b/sdk/typescript/src/semantic-convention.ts@@ -73,10 +73,11 @@ export default class SemanticConvention {static GEN_AI_RESPONSE_SYSTEM_FINGERPRINT = 'gen_ai.response.system_fingerprint';// GenAI Content- static GEN_AI_INPUT_MESSAGES = 'gen_ai.prompt';+ static GEN_AI_INPUT_MESSAGES = 'gen_ai.input.messages';+ static GEN_AI_OUTPUT_MESSAGES = 'gen_ai.output.messages';static GEN_AI_CONTENT_PROMPT_EVENT = 'gen_ai.content.prompt';- static GEN_AI_OUTPUT_MESSAGES = 'gen_ai.completion';static GEN_AI_CONTENT_COMPLETION_EVENT = 'gen_ai.content.completion';+ static GEN_AI_SYSTEM_INSTRUCTIONS = 'gen_ai.system.instructions';static GEN_AI_CONTENT_REVISED_PROMPT = 'gen_ai.content.revised_prompt';// Tool attributesdiff --git a/sdk/typescript/src/types.ts b/sdk/typescript/src/types.tsindex 5edb3dba..3937f1a7 100644--- a/sdk/typescript/src/types.ts+++ b/sdk/typescript/src/types.ts@@ -3,7 +3,7 @@ import { Resource } from '@opentelemetry/resources';import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node';import { metrics } from '@opentelemetry/api';-export type InstrumentationType = 'openai' | 'anthropic' | 'cohere';+export type InstrumentationType = 'openai' | 'anthropic' | 'cohere' | 'groq' | 'mistral' | 'google-ai' | 'together' | 'ollama';export type OpenlitInstrumentations = Partial<Record<InstrumentationType, any>>;