Skip to content

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

[Feat]: Add OpenTelemetry Instrumentation for Google AI Studio (#401)

patcher93f52fd152eb5
diff --git a/docs/images/docs-google-ai-studio-banner.gif b/docs/images/docs-google-ai-studio-banner.gif
new file mode 100644
index 00000000..aaea509b
Binary files /dev/null and b/docs/images/docs-google-ai-studio-banner.gif differ
diff --git a/docs/latest/integrations/google-ai-studio.mdx b/docs/latest/integrations/google-ai-studio.mdx
new file mode 100644
index 00000000..80b6db03
--- /dev/null
+++ b/docs/latest/integrations/google-ai-studio.mdx
@@ -0,0 +1,78 @@
+---
+title: 'Monitor Google AI Studio using OpenTelemetry'
+sidebarTitle: 'Google AI Studio'
+---
+
+<Frame>
+ <img src="/images/docs-google-ai-studio-banner.gif" />
+</Frame>
+
+OpenLIT uses OpenTelemetry Auto-Instrumentation to help you monitor LLM applications built using models from Google AI Studio. This includes tracking performance, token usage, costs, and how users interact with the application.
+
+Auto-instrumentation means you don't have to set up monitoring manually for different LLMs, frameworks, or databases. By simply adding OpenLIT in your application, all the necessary monitoring configurations are automatically set up.
+
+The integration is compatible with
+- google-generativeai `>= 0.7.2`
+
+## Get Started
+
+<Steps>
+ <Step title="Install OpenLIT">
+ Open your command line or terminal and run:
+ ```shell
+ pip install openlit
+ ```
+ </Step>
+ <Step title="Initialize OpenLIT in your Application">
+ You can set up OpenLIT in your application using either function arguments directly in your code or by using environment variables.
+ <Tabs>
+ <Tab title="Setup using function arguments">
+
+ Add the following two lines to your application code:
+
+ ```python
+ import openlit
+
+ openlit.init(
+ otlp_endpoint="YOUR_OTEL_ENDPOINT",
+ )
+ ```
+
+ Replace:
+ 1. `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.
+ </Tab>
+ <Tab title="Setup using Environment Variables">
+
+ Add the following two lines to your application code:
+
+ ```python
+ import openlit
+
+ openlit.init()
+ ```
+
+ Then, configure the your OTLP endpoint using environment variable:
+ ```shell
+ export OTEL_EXPORTER_OTLP_ENDPOINT = "YOUR_OTEL_ENDPOINT"
+ ```
+
+ Replace:
+ 1. `YOUR_OTEL_ENDPOINT` with the URL of your OpenTelemetry backend, such as `http://127.0.0.1:4318` if you are using OpenLIT and a local OTel Collector.
+ </Tab>
+ </Tabs>
+ To send metrics and traces to other Observability tools, refer to the [Connections Guide](/latest/connections/intro).
+
+ For more advanced configurations and application use cases, visit the [OpenLIT Python repository](https://github.com/openlit/openlit/tree/main/sdk/python).
+ </Step>
+</Steps>
+
+---
+
+<CardGroup cols={2}>
+<Card title="Connections" href="/latest/connections/intro" icon='link'>
+Connect to your existing Observablity Stack
+</Card>
+<Card title="SDK configuration" href="/latest/sdk-configuration" icon='file-code'>
+Documentation of the configuration options for the OpenLIT SDK.
+</Card>
+</CardGroup>
\ No newline at end of file
diff --git a/docs/latest/integrations/introduction.mdx b/docs/latest/integrations/introduction.mdx
index e28d5133..92110885 100644
--- a/docs/latest/integrations/introduction.mdx
+++ b/docs/latest/integrations/introduction.mdx
@@ -399,6 +399,22 @@ Start integrating and monitoring your applications with OpenLIT effortlessly. Ch
iconType="duotone"
>
</Card>
+ <Card
+ title="Google AI Studio"
+ href="/latest/integrations/google-ai-studio"
+ icon={
+ <svg version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 205 205">
+ <path d="M0 0 C8.04946485 6.85473908 11.51216683 14.78808209 13.04296875 25.04296875 C13.72284084 35.15606611 9.7120795 45.22637136 3.10546875 52.79296875 C-4.50478023 59.69884669 -13.19767256 61.98589369 -22.89453125 64.35546875 C-34.66881203 67.26248127 -44.54223956 70.68764928 -53.89453125 78.79296875 C-53.64733504 82.41851319 -53.47317708 84.2474189 -50.77734375 86.7890625 C-47.62185567 88.98251153 -44.4532783 90.51488513 -40.95703125 92.10546875 C-33.36614213 95.75877472 -28.99803821 99.34129857 -26.01953125 107.29296875 C-23.9433434 114.60905926 -24.61486873 120.97595594 -27.89453125 127.79296875 C-32.12953559 134.45133668 -37.27163058 137.88424244 -44.89453125 139.79296875 C-51.59470193 140.85862661 -57.6032759 139.17132116 -63.03515625 135.24609375 C-74.89986413 125.31489382 -81.03911654 108.34091202 -82.6953125 93.24169922 C-85.0510931 64.28522937 -80.12287701 37.80526376 -61.515625 14.859375 C-45.95268612 -3.09036828 -21.6056458 -15.48723283 0 0 Z " fill="#0D5ED0" transform="translate(190.89453125,46.20703125)"/>
+ <path d="M0 0 C15.07697101 12.23417513 24.16302983 28.76270314 27.41943359 47.80615234 C27.54962891 48.53189453 27.67982422 49.25763672 27.81396484 50.00537109 C30.60467234 68.34962173 26.42543136 86.9709579 16.29052734 102.38037109 C13.04453102 106.58864608 9.5020167 109.11195798 4.41943359 110.80615234 C-1.5633288 111.39269768 -5.68297763 110.11734823 -10.58056641 106.80615234 C-15.52138184 101.48527419 -15.9568107 97.88536322 -15.83837891 90.77099609 C-15.47236842 86.56187554 -14.51517635 84.87725578 -11.58056641 81.80615234 C-9.75850995 80.31311671 -7.92384837 78.83540275 -6.08056641 77.36865234 C-1.56726272 73.76524032 1.35201026 71.2208325 3.41943359 65.80615234 C3.69759141 63.04409402 3.61509714 60.59435783 3.41943359 57.80615234 C-3.75922154 53.8180106 -10.187634 52.21543176 -18.35400391 51.45458984 C-29.26661583 50.43494061 -38.78584087 48.24671604 -46.64306641 39.93115234 C-53.04862723 31.60392327 -53.73413155 23.19922494 -52.58056641 12.80615234 C-50.28248956 5.10504149 -45.34440866 -1.4669013 -38.34228516 -5.47509766 C-25.16962991 -11.67399424 -11.46370004 -8.65886518 0 0 Z " fill="#0D5FD0" transform="translate(74.58056640625,66.19384765625)"/>
+ <path d="M0 0 C0.2475 0.556875 0.495 1.11375 0.75 1.6875 C4.07335006 7.83569761 7.70469272 10.13976727 13.875 12.875 C21.86387597 16.66522519 26.67147691 20.00994553 29.875 28.5 C31.95118785 35.81609051 31.27966252 42.18298719 28 49 C23.76499566 55.65836793 18.62290067 59.09127369 11 61 C4.29982932 62.06565786 -1.70874465 60.37835241 -7.140625 56.453125 C-13.69414 50.96759023 -18.4483245 44.21858674 -21 36 C-20.67 35.01 -20.34 34.02 -20 33 C-19.8453125 33.9590625 -19.8453125 33.9590625 -19.6875 34.9375 C-19.3471875 35.9584375 -19.3471875 35.9584375 -19 37 C-18.01 37.33 -17.02 37.66 -16 38 C-15.67 39.98 -15.34 41.96 -15 44 C-14.99742188 42.88625 -14.99484375 41.7725 -14.9921875 40.625 C-14.69527161 24.7004696 -11.19120715 11.81609689 0 0 Z " fill="#8EAEFE" transform="translate(135,125)"/>
+ <path d="M0 0 C9.53843272 -0.37243338 20.25997378 -0.31915184 28 6 C37.11156809 14.87490398 41.7292091 24.92323358 45 37 C45.66 37 46.32 37 47 37 C46.18682587 44.79485489 41.70346121 51.53759248 36.01953125 56.8515625 C31.05157575 60.47425073 27.08524026 61.49879019 21 61 C14.94098975 58.92004126 10.91301855 55.8260371 8 50 C7.51801325 44.72184999 7.27631227 39.75752734 9.3125 34.8125 C11.90078663 32.10382795 14.57174 29.89259741 17.5 27.5625 C22.01330369 23.95908798 24.93257667 21.41468016 27 16 C27.27815782 13.23794168 27.19566354 10.78820549 27 8 C18.29989657 3.16660921 9.80929858 1.94928696 0 1 C0 0.67 0 0.34 0 0 Z " fill="#92B0FE" transform="translate(51,116)"/>
+ <path d="M0 0 C1.65 1.65 3.3 3.3 5 5 C4.01 5.66 3.02 6.32 2 7 C1.34 7.99 0.68 8.98 0 10 C0 6.7 0 3.4 0 0 Z " fill="#83A6FB" transform="translate(80,124)"/>
+ <path d="M0 0 C0.268125 0.639375 0.53625 1.27875 0.8125 1.9375 C1.88667752 4.33131686 1.88667752 4.33131686 5 5 C5 6.32 5 7.64 5 9 C4.01 8.67 3.02 8.34 2 8 C0.5522078 5.10441559 0 3.25811062 0 0 Z " fill="#7DA6E4" transform="translate(114,158)"/>
+ </svg>
+ }
+ iconType="duotone"
+ >
+ </Card>
<Card
title="Groq"
href="/latest/integrations/groq"
diff --git a/docs/mint.json b/docs/mint.json
index 1be26701..705dcf9a 100644
--- a/docs/mint.json
+++ b/docs/mint.json
@@ -75,6 +75,7 @@
"latest/integrations/huggingface",
"latest/integrations/bedrock",
"latest/integrations/vertexai",
+ "latest/integrations/google-ai-studio",
"latest/integrations/groq",
"latest/integrations/elevenlabs",
"latest/integrations/krutrim"
diff --git a/sdk/python/README.md b/sdk/python/README.md
index 544f3c49..b4ecc100 100644
--- a/sdk/python/README.md
+++ b/sdk/python/README.md
@@ -33,7 +33,7 @@ This project adheres to the [Semantic Conventions](https://github.com/open-telem
| [✅ Ollama](https://docs.openlit.io/latest/integrations/ollama) | [✅ Pinecone](https://docs.openlit.io/latest/integrations/pinecone) | [✅ LiteLLM](https://docs.openlit.io/latest/integrations/litellm) | |
| [✅ Anthropic](https://docs.openlit.io/latest/integrations/anthropic) | [✅ Qdrant](https://docs.openlit.io/latest/integrations/qdrant) | [✅ LlamaIndex](https://docs.openlit.io/latest/integrations/llama-index) | |
| [✅ GPT4All](https://docs.openlit.io/latest/integrations/gpt4all) | [✅ Milvus](https://docs.openlit.io/latest/integrations/milvus) | [✅ Haystack](https://docs.openlit.io/latest/integrations/haystack) | |
-| [✅ Cohere](https://docs.openlit.io/latest/integrations/cohere) | | [✅ EmbedChain](https://docs.openlit.io/latest/integrations/embedchain) | |
+| [✅ Cohere](https://docs.openlit.io/latest/integrations/cohere) | | [✅ EmbedChain](https://docs.openlit.io/latest/integrations/embedchain) | |
| [✅ Mistral](https://docs.openlit.io/latest/integrations/mistral) | | [✅ Guardrails](https://docs.openlit.io/latest/integrations/guardrails) | |
| [✅ Azure OpenAI](https://docs.openlit.io/latest/integrations/azure-openai) | | | |
| [✅ HuggingFace Transformers](https://docs.openlit.io/latest/integrations/huggingface) | | | |
@@ -43,6 +43,7 @@ This project adheres to the [Semantic Conventions](https://github.com/open-telem
| [✅ ElevenLabs](https://docs.openlit.io/latest/integrations/elevenlabs) | | | |
| [✅ vLLM](https://docs.openlit.io/latest/integrations/vllm) | | | |
| [✅ OLA Krutrim](https://docs.openlit.io/latest/integrations/krutrim) | | | |
+| [✅ Google AI Studio](https://docs.openlit.io/latest/integrations/google-ai-studio) | | | |
## Supported Destinations
- [✅ OpenTelemetry Collector](https://docs.openlit.io/latest/connections/otelcol)
diff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.toml
index 8819a15c..6628d8bf 100644
--- a/sdk/python/pyproject.toml
+++ b/sdk/python/pyproject.toml
@@ -1,6 +1,6 @@
[tool.poetry]
name = "openlit"
-version = "1.19.0"
+version = "1.20.0"
description = "OpenTelemetry-native Auto instrumentation library for monitoring LLM Applications and GPUs, facilitating the integration of observability into your GenAI-driven projects"
authors = ["OpenLIT"]
repository = "https://github.com/openlit/openlit/tree/main/openlit/python"
diff --git a/sdk/python/src/openlit/__init__.py b/sdk/python/src/openlit/__init__.py
index c51c04de..ba257b74 100644
--- a/sdk/python/src/openlit/__init__.py
+++ b/sdk/python/src/openlit/__init__.py
@@ -33,6 +33,7 @@ from openlit.instrumentation.ollama import OllamaInstrumentor
from openlit.instrumentation.gpt4all import GPT4AllInstrumentor
from openlit.instrumentation.elevenlabs import ElevenLabsInstrumentor
from openlit.instrumentation.vllm import VLLMInstrumentor
+from openlit.instrumentation.google_ai_studio import GoogleAIStudioInstrumentor
from openlit.instrumentation.langchain import LangChainInstrumentor
from openlit.instrumentation.llamaindex import LlamaIndexInstrumentor
from openlit.instrumentation.haystack import HaystackInstrumentor
@@ -196,6 +197,7 @@ def init(environment="default", application_name="default", tracer=None, otlp_en
"gpt4all": "gpt4all",
"elevenlabs": "elevenlabs",
"vllm": "vllm",
+ "google-ai-studio": "google.generativeai",
"langchain": "langchain",
"llama_index": "llama_index",
"haystack": "haystack",
@@ -273,6 +275,7 @@ def init(environment="default", application_name="default", tracer=None, otlp_en
"gpt4all": GPT4AllInstrumentor(),
"elevenlabs": ElevenLabsInstrumentor(),
"vllm": VLLMInstrumentor(),
+ "google-ai-studio": GoogleAIStudioInstrumentor(),
"langchain": LangChainInstrumentor(),
"llama_index": LlamaIndexInstrumentor(),
"haystack": HaystackInstrumentor(),
diff --git a/sdk/python/src/openlit/instrumentation/google_ai_studio/__init__.py b/sdk/python/src/openlit/instrumentation/google_ai_studio/__init__.py
new file mode 100644
index 00000000..3e966409
--- /dev/null
+++ b/sdk/python/src/openlit/instrumentation/google_ai_studio/__init__.py
@@ -0,0 +1,56 @@
+# pylint: disable=useless-return, bad-staticmethod-argument, disable=duplicate-code
+"""Initializer of Auto Instrumentation of Google AI Studio Functions"""
+
+from typing import Collection
+import importlib.metadata
+from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
+from wrapt import wrap_function_wrapper
+
+from openlit.instrumentation.google_ai_studio.google_ai_studio import (
+ generate
+)
+
+from openlit.instrumentation.google_ai_studio.async_google_ai_studio import (
+ async_generate
+)
+
+_instruments = ("google-generativeai >= 0.2.0",)
+
+class GoogleAIStudioInstrumentor(BaseInstrumentor):
+ """
+ An instrumentor for google-generativeai's client library.
+ """
+
+ def instrumentation_dependencies(self) -> Collection[str]:
+ return _instruments
+
+ def _instrument(self, **kwargs):
+ application_name = kwargs.get("application_name", "default_application")
+ environment = kwargs.get("environment", "default_environment")
+ tracer = kwargs.get("tracer")
+ metrics = kwargs.get("metrics_dict")
+ pricing_info = kwargs.get("pricing_info", {})
+ trace_content = kwargs.get("trace_content", False)
+ disable_metrics = kwargs.get("disable_metrics")
+ version = importlib.metadata.version("ollama")
+
+ # sync generate
+ wrap_function_wrapper(
+ "google.generativeai.generative_models",
+ "GenerativeModel.generate_content",
+ generate("google_ai_studio.generate_content", version, environment, application_name,
+ tracer, pricing_info, trace_content, metrics, disable_metrics),
+ )
+
+ # async generate
+ wrap_function_wrapper(
+ "google.generativeai.generative_models",
+ "GenerativeModel.generate_content_async",
+ async_generate("google_ai_studio.generate_content", version, environment,
+ application_name, tracer, pricing_info, trace_content, metrics,
+ disable_metrics),
+ )
+
+ def _uninstrument(self, **kwargs):
+ # Proper uninstrumentation logic to revert patched methods
+ pass
diff --git a/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.py b/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.py
new file mode 100644
index 00000000..39ab49a2
--- /dev/null
+++ b/sdk/python/src/openlit/instrumentation/google_ai_studio/async_google_ai_studio.py
@@ -0,0 +1,259 @@
+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment, protected-access
+"""
+Module for monitoring Ollama API calls.
+"""
+
+import logging
+from opentelemetry.trace import SpanKind, Status, StatusCode
+from opentelemetry.sdk.resources import TELEMETRY_SDK_NAME
+from openlit.__helpers import (
+ handle_exception,
+ get_chat_model_cost,
+)
+from openlit.semcov import SemanticConvetion
+
+# Initialize logger for logging potential issues and operations
+logger = logging.getLogger(__name__)
+
+def async_generate(gen_ai_endpoint, version, environment, application_name,
+ tracer, pricing_info, trace_content, metrics, disable_metrics):
+ """
+ Generates a telemetry wrapper for chat to collect metrics.
+
+ Args:
+ gen_ai_endpoint: Endpoint identifier for logging and tracing.
+ version: Version of the monitoring package.
+ environment: Deployment environment (e.g., production, staging).
+ application_name: Name of the application using the Ollama API.
+ tracer: OpenTelemetry tracer for creating spans.
+ pricing_info: Information used for calculating the cost of Ollama usage.
+ trace_content: Flag indicating whether to trace the actual content.
+
+ Returns:
+ A function that wraps the chat method to add telemetry.
+ """
+
+ async def wrapper(wrapped, instance, args, kwargs):
+ """
+ Wraps the 'chat' API call to add telemetry.
+
+ This collects metrics such as execution time, cost, and token usage, and handles errors
+ gracefully, adding details to the trace for observability.
+
+ Args:
+ wrapped: The original 'chat' method to be wrapped.
+ instance: The instance of the class where the original method is defined.
+ args: Positional arguments for the 'chat' method.
+ kwargs: Keyword arguments for the 'chat' method.
+
+ Returns:
+ The response from the original 'chat' method.
+ """
+ # pylint: disable=no-else-return
+ if kwargs.get('stream', False) is True:
+ # Special handling for streaming response to accommodate the nature of data flow
+ async def stream_generator():
+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:
+ # Placeholder for aggregating streaming response
+ llmresponse = ""
+
+ # Loop through streaming events capturing relevant details
+ async for chunk in await wrapped(*args, **kwargs):
+ # Collect message IDs and aggregated response from events
+ content = chunk.text
+ if content:
+ llmresponse += content
+
+ input_tokens = chunk.usage_metadata.prompt_token_count
+ output_tokens = chunk.usage_metadata.candidates_token_count
+ yield chunk
+
+ # Handling exception ensure observability without disrupting operation
+ try:
+ prompt = ""
+ for arg in args:
+ if isinstance(arg, str):
+ prompt = f"{prompt}{arg}\n"
+ elif isinstance(arg, list):
+ for subarg in arg:
+ prompt = f"{prompt}{subarg}\n"
+ if hasattr(instance, "_model_id"):
+ model = instance._model_id
+ if hasattr(instance, "_model_name"):
+ model = instance._model_name.replace("publishers/google/models/", "")
+
+ total_tokens = input_tokens + output_tokens
+ # Calculate cost of the operation
+ cost = get_chat_model_cost(kwargs.get("model", "gpt-3.5-turbo"),
+ pricing_info, input_tokens,
+ output_tokens)
+
+ # Set Span attributes
+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")
+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,
+ SemanticConvetion.GEN_AI_SYSTEM_GOOGLE_AI_STUDIO)
+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,
+ SemanticConvetion.GEN_AI_TYPE_CHAT)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,
+ gen_ai_endpoint)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
+ environment)
+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,
+ application_name)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_MODEL,
+ model)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_IS_STREAM,
+ True)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_PROMPT_TOKENS,
+ input_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COMPLETION_TOKENS,
+ output_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_TOTAL_TOKENS,
+ total_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COST,
+ cost)
+ if trace_content:
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_PROMPT_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_PROMPT: prompt,
+ },
+ )
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_COMPLETION_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_COMPLETION: llmresponse,
+ },
+ )
+
+ span.set_status(Status(StatusCode.OK))
+
+ if disable_metrics is False:
+ attributes = {
+ TELEMETRY_SDK_NAME:
+ "openlit",
+ SemanticConvetion.GEN_AI_APPLICATION_NAME:
+ application_name,
+ SemanticConvetion.GEN_AI_SYSTEM:
+ SemanticConvetion.GEN_AI_SYSTEM_GOOGLE_AI_STUDIO,
+ SemanticConvetion.GEN_AI_ENVIRONMENT:
+ environment,
+ SemanticConvetion.GEN_AI_TYPE:
+ SemanticConvetion.GEN_AI_TYPE_CHAT,
+ SemanticConvetion.GEN_AI_REQUEST_MODEL:
+ model
+ }
+
+ metrics["genai_requests"].add(1, attributes)
+ metrics["genai_total_tokens"].add(
+ total_tokens, attributes
+ )
+ metrics["genai_completion_tokens"].add(output_tokens, attributes)
+ metrics["genai_prompt_tokens"].add(input_tokens, attributes)
+ metrics["genai_cost"].record(cost, attributes)
+
+ except Exception as e:
+ handle_exception(span, e)
+ logger.error("Error in trace creation: %s", e)
+
+ return stream_generator()
+ else:
+ # pylint: disable=line-too-long
+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:
+ response = await wrapped(*args, **kwargs)
+
+ try:
+ prompt = ""
+ for arg in args:
+ if isinstance(arg, str):
+ prompt = f"{prompt}{arg}\n"
+ elif isinstance(arg, list):
+ for subarg in arg:
+ prompt = f"{prompt}{subarg}\n"
+ if hasattr(instance, "_model_id"):
+ model = instance._model_id
+ if hasattr(instance, "_model_name"):
+ model = instance._model_name.replace("publishers/google/models/", "")
+
+ # Set base span attribues
+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")
+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,
+ SemanticConvetion.GEN_AI_SYSTEM_GOOGLE_AI_STUDIO)
+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,
+ SemanticConvetion.GEN_AI_TYPE_CHAT)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,
+ gen_ai_endpoint)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
+ environment)
+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,
+ application_name)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_MODEL,
+ model)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_IS_STREAM,
+ False)
+
+ if trace_content:
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_PROMPT_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_PROMPT: prompt,
+ },
+ )
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_COMPLETION_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_COMPLETION: response.text,
+ },
+ )
+
+ prompt_tokens = response.usage_metadata.prompt_token_count
+ completion_tokens = response.usage_metadata.candidates_token_count
+ total_tokens = response.usage_metadata.total_token_count
+ # Calculate cost of the operation
+ cost = get_chat_model_cost(kwargs.get("model", "llama3"),
+ pricing_info, prompt_tokens, completion_tokens)
+
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_PROMPT_TOKENS,
+ prompt_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COMPLETION_TOKENS,
+ completion_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_TOTAL_TOKENS,
+ total_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COST,
+ cost)
+
+ span.set_status(Status(StatusCode.OK))
+
+ if disable_metrics is False:
+ attributes = {
+ TELEMETRY_SDK_NAME:
+ "openlit",
+ SemanticConvetion.GEN_AI_APPLICATION_NAME:
+ application_name,
+ SemanticConvetion.GEN_AI_SYSTEM:
+ SemanticConvetion.GEN_AI_SYSTEM_OLLAMA,
+ SemanticConvetion.GEN_AI_ENVIRONMENT:
+ environment,
+ SemanticConvetion.GEN_AI_TYPE:
+ SemanticConvetion.GEN_AI_TYPE_CHAT,
+ SemanticConvetion.GEN_AI_REQUEST_MODEL:
+ kwargs.get("model", "llama3")
+ }
+
+ metrics["genai_requests"].add(1, attributes)
+ metrics["genai_total_tokens"].add(total_tokens, attributes)
+ metrics["genai_completion_tokens"].add(completion_tokens, attributes)
+ metrics["genai_prompt_tokens"].add(prompt_tokens, attributes)
+ metrics["genai_cost"].record(cost, attributes)
+
+ # Return original response
+ return response
+
+ except Exception as e:
+ handle_exception(span, e)
+ logger.error("Error in trace creation: %s", e)
+
+ # Return original response
+ return response
+
+ return wrapper
diff --git a/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.py b/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.py
new file mode 100644
index 00000000..0af908ab
--- /dev/null
+++ b/sdk/python/src/openlit/instrumentation/google_ai_studio/google_ai_studio.py
@@ -0,0 +1,261 @@
+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment, protected-access
+"""
+Module for monitoring Ollama API calls.
+"""
+
+import logging
+from opentelemetry.trace import SpanKind, Status, StatusCode
+from opentelemetry.sdk.resources import TELEMETRY_SDK_NAME
+from openlit.__helpers import (
+ handle_exception,
+ get_chat_model_cost,
+)
+from openlit.semcov import SemanticConvetion
+
+# Initialize logger for logging potential issues and operations
+logger = logging.getLogger(__name__)
+
+def generate(gen_ai_endpoint, version, environment, application_name,
+ tracer, pricing_info, trace_content, metrics, disable_metrics):
+ """
+ Generates a telemetry wrapper for chat to collect metrics.
+
+ Args:
+ gen_ai_endpoint: Endpoint identifier for logging and tracing.
+ version: Version of the monitoring package.
+ environment: Deployment environment (e.g., production, staging).
+ application_name: Name of the application using the Ollama API.
+ tracer: OpenTelemetry tracer for creating spans.
+ pricing_info: Information used for calculating the cost of Ollama usage.
+ trace_content: Flag indicating whether to trace the actual content.
+
+ Returns:
+ A function that wraps the chat method to add telemetry.
+ """
+
+ def wrapper(wrapped, instance, args, kwargs):
+ """
+ Wraps the 'chat' API call to add telemetry.
+
+ This collects metrics such as execution time, cost, and token usage, and handles errors
+ gracefully, adding details to the trace for observability.
+
+ Args:
+ wrapped: The original 'chat' method to be wrapped.
+ instance: The instance of the class where the original method is defined.
+ args: Positional arguments for the 'chat' method.
+ kwargs: Keyword arguments for the 'chat' method.
+
+ Returns:
+ The response from the original 'chat' method.
+ """
+ # pylint: disable=no-else-return
+ if kwargs.get("stream", False) is True:
+ # Special handling for streaming response to accommodate the nature of data flow
+ def stream_generator():
+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:
+ # Placeholder for aggregating streaming response
+ llmresponse = ""
+
+ # Loop through streaming events capturing relevant details
+ for chunk in wrapped(*args, **kwargs):
+ # Collect message IDs and aggregated response from events
+ content = chunk.text
+ if content:
+ llmresponse += content
+
+ input_tokens = chunk.usage_metadata.prompt_token_count
+ output_tokens = chunk.usage_metadata.candidates_token_count
+ yield chunk
+
+ # Handling exception ensure observability without disrupting operation
+ try:
+ prompt = ""
+ for arg in args:
+ if isinstance(arg, str):
+ prompt = f"{prompt}{arg}\n"
+ elif isinstance(arg, list):
+ for subarg in arg:
+ prompt = f"{prompt}{subarg}\n"
+ if hasattr(instance, "_model_id"):
+ model = instance._model_id
+ if hasattr(instance, "_model_name"):
+ model = instance._model_name.replace("publishers/google/models/", "")
+
+ total_tokens = input_tokens + output_tokens
+ # Calculate cost of the operation
+ cost = get_chat_model_cost(kwargs.get("model", "gpt-3.5-turbo"),
+ pricing_info, input_tokens,
+ output_tokens)
+
+ # Set Span attributes
+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")
+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,
+ SemanticConvetion.GEN_AI_SYSTEM_GOOGLE_AI_STUDIO)
+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,
+ SemanticConvetion.GEN_AI_TYPE_CHAT)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,
+ gen_ai_endpoint)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
+ environment)
+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,
+ application_name)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_MODEL,
+ model)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_IS_STREAM,
+ True)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_PROMPT_TOKENS,
+ input_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COMPLETION_TOKENS,
+ output_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_TOTAL_TOKENS,
+ total_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COST,
+ cost)
+ if trace_content:
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_PROMPT_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_PROMPT: prompt,
+ },
+ )
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_COMPLETION_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_COMPLETION: llmresponse,
+ },
+ )
+
+ span.set_status(Status(StatusCode.OK))
+
+ if disable_metrics is False:
+ attributes = {
+ TELEMETRY_SDK_NAME:
+ "openlit",
+ SemanticConvetion.GEN_AI_APPLICATION_NAME:
+ application_name,
+ SemanticConvetion.GEN_AI_SYSTEM:
+ SemanticConvetion.GEN_AI_SYSTEM_GOOGLE_AI_STUDIO,
+ SemanticConvetion.GEN_AI_ENVIRONMENT:
+ environment,
+ SemanticConvetion.GEN_AI_TYPE:
+ SemanticConvetion.GEN_AI_TYPE_CHAT,
+ SemanticConvetion.GEN_AI_REQUEST_MODEL:
+ model
+ }
+
+ metrics["genai_requests"].add(1, attributes)
+ metrics["genai_total_tokens"].add(
+ total_tokens, attributes
+ )
+ metrics["genai_completion_tokens"].add(output_tokens, attributes)
+ metrics["genai_prompt_tokens"].add(input_tokens, attributes)
+ metrics["genai_cost"].record(cost, attributes)
+
+ except Exception as e:
+ handle_exception(span, e)
+ logger.error("Error in trace creation: %s", e)
+
+ return stream_generator()
+ else:
+ # pylint: disable=line-too-long
+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:
+ response = wrapped(*args, **kwargs)
+
+ # print(instance._system_instruction.__dict__["_pb"].parts[0].text)
+ try:
+ prompt = ""
+ for arg in args:
+ if isinstance(arg, str):
+ prompt = f"{prompt}{arg}\n"
+ elif isinstance(arg, list):
+ for subarg in arg:
+ prompt = f"{prompt}{subarg}\n"
+
+ if hasattr(instance, "_model_id"):
+ model = instance._model_id
+ if hasattr(instance, "_model_name"):
+ model = instance._model_name.replace("publishers/google/models/", "")
+
+ # Set base span attribues
+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")
+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,
+ SemanticConvetion.GEN_AI_SYSTEM_GOOGLE_AI_STUDIO)
+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,
+ SemanticConvetion.GEN_AI_TYPE_CHAT)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,
+ gen_ai_endpoint)
+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,
+ environment)
+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,
+ application_name)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_MODEL,
+ model)
+ span.set_attribute(SemanticConvetion.GEN_AI_REQUEST_IS_STREAM,
+ False)
+
+ if trace_content:
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_PROMPT_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_PROMPT: prompt,
+ },
+ )
+ span.add_event(
+ name=SemanticConvetion.GEN_AI_CONTENT_COMPLETION_EVENT,
+ attributes={
+ SemanticConvetion.GEN_AI_CONTENT_COMPLETION: response.text,
+ },
+ )
+
+ prompt_tokens = response.usage_metadata.prompt_token_count
+ completion_tokens = response.usage_metadata.candidates_token_count
+ total_tokens = response.usage_metadata.total_token_count
+ # Calculate cost of the operation
+ cost = get_chat_model_cost(kwargs.get("model", "llama3"),
+ pricing_info, prompt_tokens, completion_tokens)
+
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_PROMPT_TOKENS,
+ prompt_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COMPLETION_TOKENS,
+ completion_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_TOTAL_TOKENS,
+ total_tokens)
+ span.set_attribute(SemanticConvetion.GEN_AI_USAGE_COST,
+ cost)
+
+ span.set_status(Status(StatusCode.OK))
+
+ if disable_metrics is False:
+ attributes = {
+ TELEMETRY_SDK_NAME:
+ "openlit",
+ SemanticConvetion.GEN_AI_APPLICATION_NAME:
+ application_name,
+ SemanticConvetion.GEN_AI_SYSTEM:
+ SemanticConvetion.GEN_AI_SYSTEM_OLLAMA,
+ SemanticConvetion.GEN_AI_ENVIRONMENT:
+ environment,
+ SemanticConvetion.GEN_AI_TYPE:
+ SemanticConvetion.GEN_AI_TYPE_CHAT,
+ SemanticConvetion.GEN_AI_REQUEST_MODEL:
+ kwargs.get("model", "llama3")
+ }
+
+ metrics["genai_requests"].add(1, attributes)
+ metrics["genai_total_tokens"].add(total_tokens, attributes)
+ metrics["genai_completion_tokens"].add(completion_tokens, attributes)
+ metrics["genai_prompt_tokens"].add(prompt_tokens, attributes)
+ metrics["genai_cost"].record(cost, attributes)
+
+ # Return original response
+ return response
+
+ except Exception as e:
+ handle_exception(span, e)
+ logger.error("Error in trace creation: %s", e)
+
+ # Return original response
+ return response
+
+ return wrapper
diff --git a/sdk/python/src/openlit/semcov/__init__.py b/sdk/python/src/openlit/semcov/__init__.py
index 855cf2be..e127d667 100644
--- a/sdk/python/src/openlit/semcov/__init__.py
+++ b/sdk/python/src/openlit/semcov/__init__.py
@@ -102,6 +102,7 @@ class SemanticConvetion:
GEN_AI_SYSTEM_GPT4ALL = "gpt4all"
GEN_AI_SYSTEM_ELEVENLABS = "elevenlabs"
GEN_AI_SYSTEM_VLLM = "vLLM"
+ GEN_AI_SYSTEM_GOOGLE_AI_STUDIO = "google-ai-studio"
GEN_AI_SYSTEM_LANGCHAIN = "langchain"
GEN_AI_SYSTEM_LLAMAINDEX = "llama_index"
GEN_AI_SYSTEM_HAYSTACK = "haystack"
diff --git a/sdk/python/tests/requirements.txt b/sdk/python/tests/requirements.txt
index 93d77488..865f4a6c 100644
--- a/sdk/python/tests/requirements.txt
+++ b/sdk/python/tests/requirements.txt
@@ -16,6 +16,7 @@ botocore>=1.34.93
google-cloud-aiplatform >= 1.38.1
groq >= 0.5.0
elevenlabs >= 1.4.0
+google-generativeai >= 0.7.2
# Vector DBs
chromadb>=0.4.0
diff --git a/sdk/python/tests/test_google_ai_studio.py b/sdk/python/tests/test_google_ai_studio.py
new file mode 100644
index 00000000..31c601bc
--- /dev/null
+++ b/sdk/python/tests/test_google_ai_studio.py
@@ -0,0 +1,73 @@
+# pylint: disable=duplicate-code, no-name-in-module
+"""
+This module contains tests for Google AI Studio functionality using the google-generativeai
+Python library.
+
+Tests cover various API endpoints, including chat.
+These tests validate integration with OpenLIT.
+
+Environment Variables:
+ - GOOGLE_AI_STUDIO_API_TOKEN: Google AI Studio API api_key for authentication.
+
+Note: Ensure the environment is properly configured for Google AI Studio access and
+OpenLIT monitoring prior to running these tests.
+"""
+
+import os
+import pytest
+import google.generativeai as genai
+import openlit
+
+# Initialize Google AI Studio client
+genai.configure(api_key=os.getenv("GOOGLE_AI_STUDIO_API_TOKEN"))
+model = genai.GenerativeModel("gemini-1.5-flash")
+
+# Initialize environment and application name for OpenLIT monitoring
+openlit.init(environment="openlit-testing", application_name="openlit-python-test")
+
+def test_sync_generate_content():
+ """
+ Tests synchronous Generate content with the "gemini-1.5-flash" model.
+
+ Raises:
+ AssertionError: If the generate content response object is not as expected.
+ """
+
+ try:
+ response = model.generate_content("Observability for LLMs", stream=False)
+ assert isinstance(response.text, str)
+
+ response = model.generate_content("Monitor AI", stream=True)
+ for text in response:
+ assert isinstance(text.text, str)
+
+ # pylint: disable=broad-exception-caught
+ except Exception as e:
+ if "rate limit" in str(e).lower():
+ print("Rate Limited:", e)
+ else:
+ raise
+
+async def test_async_generate_content():
+ """
+ Tests synchronous Generate content with the "gemini-1.5-flash" model.
+
+ Raises:
+ AssertionError: If the generate content response object is not as expected.
+ """
+
+ try:
+ response = await model.generate_content_async("Observability for LLMs", stream=False)
+ assert isinstance(response.text, str)
+
+ response = await model.generate_content_async("Monitor AI", stream=True)
+ async for text in response:
+ assert isinstance(text.text, str)
+
+ # pylint: disable=broad-exception-caught
+ except Exception as e:
+ if "rate limit" in str(e).lower():
+ print("Rate Limited:", e)
+ else:
+ raise