Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
[Feat]: Support for Monitoring Haystack and LlamaIndex Frameworks (#227)
patcher94339e552237f
diff --git a/assets/pricing.json b/assets/pricing.jsonindex 3927230d..9dcad1ab 100644--- a/assets/pricing.json+++ b/assets/pricing.json@@ -131,6 +131,14 @@"promptPrice": 0.01,"completionPrice": 0.03},+ "gpt-4o": {+ "promptPrice": 0.005,+ "completionPrice": 0.015+ },+ "gpt-4o-2024-05-13": {+ "promptPrice": 0.005,+ "completionPrice": 0.015+ },"azure_gpt-4": {"promptPrice": 0.03,"completionPrice": 0.06diff --git a/docs/images/docs-haystack-banner.gif b/docs/images/docs-haystack-banner.gifnew file mode 100644index 00000000..cffe2d18Binary files /dev/null and b/docs/images/docs-haystack-banner.gif differdiff --git a/docs/images/docs-llama-index-banner.gif b/docs/images/docs-llama-index-banner.gifnew file mode 100644index 00000000..74544244Binary files /dev/null and b/docs/images/docs-llama-index-banner.gif differdiff --git a/docs/latest/integrations/haystack.mdx b/docs/latest/integrations/haystack.mdxnew file mode 100644index 00000000..b4b4d1fc--- /dev/null+++ b/docs/latest/integrations/haystack.mdx@@ -0,0 +1,64 @@+---+title: 'Monitor Haystack using OpenTelemetry'+sidebarTitle: 'Haystack'+---++<Frame>+ <img src="/images/docs-haystack-banner.gif" />+</Frame>++OpenLIT SDKs provide OpenTelemetry Auto-Instrumentation for monitoring LLM Application using Haystack which includes performance and operation stats.++## Get Started++<Steps>+ <Step title="Disable Haystack default tracing">+ To disable default tracing available in Haystack, there are two options:++ Set the environment variable `HAYSTACK_AUTO_TRACE_ENABLED` to `false` when running your Haystack application++ or++ Disable tracing in Python:++ ```Python+ from haystack.tracing import disable_tracing++ disable_tracing()+ ```+ </Step>+ <Step title="Install the OpenLIT Python SDK using pip">+ ```shell+ pip install openlit+ ```+ </Step>+ <Step title="Add the following two lines to your application code:">+ Integrating the OpenLIT into LLM applications is straightforward. Start monitoring for your LLM Application with just **one line of code**:++ ```python+ import openlit++ openlit.init(otlp_endpoint="otlp_backend_endpoint", otlp_headers="otlp_backend_token")+ ```++ Alternatively, you can configure the OTLP endpoint by setting the `OTEL_EXPORTER_OTLP_ENDPOINT` and the OTLP headers by setting the Appli`OTEL_EXPORTER_OTLP_HEADERS` environmentcation variable as recommended in the OpenTelemetry documentation.++ If your aim is primarily to test or develop without sending telemetry data to an external backend, you can initialize OpenLIT without any parameters. This approach directs all telemetry output to your console, enabling quick diagnostics and local debugging:++ ```python+ import openlit++ openlit.init()+ ```++ Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.+ </Step>+</Steps>++---++<CardGroup cols={2}>+<Card title="Connections" href="/latest/connections/intro" icon='link'>+Connect to your existing Observablity Stack+</Card>+</CardGroup>\ No newline at end of filediff --git a/docs/latest/integrations/introduction.mdx b/docs/latest/integrations/introduction.mdxindex 1a7d699d..7f3ba575 100644--- a/docs/latest/integrations/introduction.mdx+++ b/docs/latest/integrations/introduction.mdx@@ -225,6 +225,29 @@ Start integrating and monitoring your applications with OpenLIT effortlessly. Ch}></Card>+ <Card+ title="LlamaIndex"+ href="/latest/integrations/llama-index"+ iconType="duotone"+ icon={+ <svg id="LlamaIndexSquare" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1620 1620">+ <path d="M1227.57,1375.94c-3.92-4.37-7.85-8.74-11.77-13.1-27.64-30.21-41.35-65.1-30.46-111.35-104.67,33.94-205.19,41.62-308.99,17.88-2.72,31.05-3.81,59.43-8.02,87.34-5.15,34.16-13.02,67.9-19.47,101.86-2.25,11.83-4.72,23.74-5.46,35.71-2.2,35.64-1.41,71.63-6.21,106.88-2.47,18.13-14.33,34.99-21.3,50.93h-77.71l.42-.97c4.15-11.29,8.29-22.58,12.44-33.87,0,0-.21.27-.21.27,4.2-4.21,8.39-8.43,12.59-12.64l-.27.27c2.12-2.13,4.24-4.25,6.36-6.38l11.44-1.32c1.11-18.89.94-11.95,1.11-18.89.83-33.72.34-67.53-1.47-101.21-.67-12.36-6.26-24.46-9.6-36.68-2.32.06-4.64.12-6.96.18-12.39,44.7-24.78,89.41-37.17,134.11,0,0,.55-.24.55-.24-8.87,16.08-21.35,47.27-30.22,63.35-11.03,1-42.13-.92-67.88-.77,2.1-21.24,9.98-32.37,23.37-44.39,28.79-25.84,36.54-63.75,36.43-100.05-.15-50.7-6.92-101.37-10.86-152.06,5.59-10.19,8.83-23.19,17.14-30.11,56.24-46.85,102.12-102.83,144.95-161.48,41.46-56.78,83.19-113.45,122.26-171.88,43.03-64.36,87.61-128.6,97.48-208.71,72.88,7.68,142.46,25.43,207.78,60.42,19.96,10.69,45.01,11.89,67.72,17.44,8.42,6.5,20.9,11.26,24.57,19.8,18.5,43.03,36.6,86.42,51.23,130.87,17.99,54.67,22.8,110.74,13.5,168.57-11.14,69.24-41.63,128.83-80.9,185.36-6.53,9.41-8.65,21.88-12.81,32.94,0,0,.15-.18.15-.18-6.82,4.82-14.03,9.18-20.38,14.54-17.18,14.49-21.33,42.61-48.67,46.55-.37-1.49-.55-2.99-.52-4.53-1.44-2.72-2.87-5.43-4.31-8.15-9.96-18.77-19.93-37.53-29.89-56.3ZM517.28,570.6c-10.46-.44-21.29,7.69-31.95,11.86,0,0-9.76,80.82-9.76,121.49s13.91,75.15,16.27,113.41c1.68,27.25,3.01,56.38,13.57,80.73,12.68,29.21,18.19,57.44,16.61,88.5-4.38,86.3,40.37,144.69,108.96,189.16,30.85-33.52,64.34-65,91.99-100.98,55.34-72.02,102.6-149.18,128.62-237.37,5.42-18.37,8.99-37.27,14.28-59.58,2.88-2.94,8.79-8.99,14.7-15.04l-141.94-.4c-2.09-1.89-3.55-3.22-5.64-5.11l-17.96-175.16c-65.91-4.05-131.8-8.74-197.77-11.52ZM665.84,361.83c-23.76-47.96-58.65-85.02-111.51-106.14-.84,14.47-1.52,26.06-2.26,38.88-17.66-12.73-33.28-23.98-53.45-38.52,2.58,25.98,5.08,44.11,6.04,62.33,1.89,36.02-13.58,54.11-49.86,59.15-7.07.98-14.41,0-21.52.83-21.28,2.46-32.65,14.43-35.51,35.63-3.74,27.68,10.41,45.81,33.06,56.83,23.57,11.47,48.7,19.75,77.45,31.13-4.21,14.19-8.49,28.22-12.51,42.32-3.62,12.69-6.95,25.46-10.41,38.2,10.66-4.17,21.48-12.3,31.95-11.86,65.97,2.78,131.86,7.46,197.77,11.52-3.72-76.27-14.91-151.03-49.22-220.28ZM1363.96,1424.58c1.96,2.05,3.91,4.09,5.87,6.14,2.05,2.06,4.1,4.12,6.15,6.17,2.04,1.96,4.07,3.92,6.11,5.88,7.5,4.99,15,9.99,22.49,14.98,3.64-19.55,25.85-47.67,44.43-61.31,7.67-5.63,11.05-16.91,16.99-25.18,3.32-4.62,7.89-8.34,11.9-12.46-1.62-13.04-4.17-26.03-4.67-39.11-1.43-37.53.92-74.57-10.84-112.04-14.13-45-8.64-92.52-6.02-140.17,43.12-8.67,61.47-36.79,63.89-74.64,2.92-45.72-5.65-89.82-34.29-127.23-20.53-26.81-48.34-41.86-82.84-36.43-22.85,3.59-45.04,11.37-67.53,17.28,8.42,6.5,20.9,11.26,24.57,19.8,18.5,43.03,36.6,86.42,51.23,130.87,17.99,54.67,22.8,110.74,13.5,168.57-11.14,69.24-41.63,128.83-80.9,185.36-6.53,9.41-8.65,21.88-12.81,32.94,10.93,13.53,21.85,27.05,32.78,40.58ZM985.74,765.38c-6.29,81.15-44.94,149.41-87.84,215.37-37.77,58.07-75.75,116.3-118.15,170.97-30.45,39.26-68.33,72.77-102.9,108.84.47,26.74.94,53.49,1.41,80.23,5.59-10.19,8.83-23.19,17.14-30.11,56.24-46.85,102.12-102.83,144.95-161.48,41.46-56.78,83.19-113.45,122.26-171.88,43.03-64.36,87.61-128.6,97.48-208.71-24.78-1.08-49.56-2.15-74.34-3.23ZM664.54,1198.77c31.99-35.25,67.97-67.72,95.05-106.41,41.85-59.77,79.26-122.86,115.33-186.36,20.46-36.01,35.34-75.24,37.26-118.13.34-7.72,8.23-15.09,12.63-22.63-3.62.1-7.23.2-10.85.3-4.64-.95-9.28-1.91-13.91-2.86-6.49.04-12.97.07-19.46.11-5.91,6.05-11.83,12.1-14.7,15.04-5.29,22.3-8.87,41.21-14.28,59.58-26.02,88.19-73.28,165.35-128.62,237.37-27.65,35.98-61.14,67.46-91.99,100.98,11.18,7.67,22.36,15.34,33.54,23.02ZM942.72,765.23c2.4,67.42-27.68,124.69-59.91,180.03-31.13,53.43-66.12,104.74-101.63,155.44-20.89,29.83-45.84,56.84-69.42,84.74-11.1,13.13-23.27,25.36-34.96,37.99.02,12.38.03,24.75.05,37.13,34.57-36.07,72.45-69.58,102.9-108.84,42.4-54.66,80.39-112.89,118.15-170.97,42.9-65.96,81.55-134.22,87.84-215.37-14.34-.05-28.68-.1-43.03-.15ZM1405.73,1462.64c5.46,11.19,15.47,22.34,15.54,33.56.25,39.8-2.64,79.65-5.17,119.42-.22,3.52-6.45,6.66-9.89,9.98-2.3,1.72-4.61,3.44-6.91,5.16-1.99,2.12-3.98,4.24-5.97,6.35-2.17,1.91-4.35,3.81-6.52,5.72,3.41,3.84,6.65,10.81,10.25,11,20.21,1.08,40.5.52,61.67.52,17.8-43.47,23.9-85.93,17.32-130.35-1.3-8.8-3.75-19.81.05-26.59,25.92-46.24,19.03-92.21,1.8-138.62-4.01,4.12-8.58,7.84-11.9,12.46-5.94,8.27-9.32,19.55-16.99,25.18-18.58,13.64-40.79,41.76-44.43,61.31.29,1.65.67,3.28,1.15,4.89ZM924.82,765.23c-4.4,7.53-12.29,14.91-12.63,22.63-1.92,42.89-16.8,82.12-37.26,118.13-36.08,63.51-73.48,126.6-115.33,186.36-27.09,38.69-63.06,71.15-95.05,106.41,4.09,8.22,8.17,16.44,12.26,24.66,11.69-12.64,23.86-24.86,34.96-37.99,23.58-27.89,48.53-54.91,69.42-84.74,35.5-50.7,70.5-102.01,101.63-155.44,32.23-55.34,62.31-112.61,59.91-180.03-5.97,0-11.93,0-17.9,0ZM900.05,762.67c4.64.95,9.28,1.91,13.91,2.86-4.64-.95-9.28-1.91-13.91-2.86ZM733.27,759.83c1.79.85,3.59,1.7,5.38,2.55-1.6-1.25-3.4-2.1-5.38-2.55ZM479.39,702.93c1.24,2.91-1.24,5.83,0,8.74-1.24-2.91,1.24-5.83,0-8.74ZM1215.35,1588.66c-8.44,5.86-16.88,11.72-25.32,17.58-1.64,9.45-3.28,18.9-5,28.8h72.24c0-14.59-.97-27.72.34-40.61.87-8.56,2.71-18.87,8.08-24.82,47.78-52.98,66.64-115.78,65.65-185.78-6.82,4.82-14.03,9.18-20.38,14.54-17.18,14.49-21.33,42.61-48.67,46.55.78,53.28-15.32,101.04-46.93,143.74ZM1257.46,1432.24c1.44,2.72,2.87,5.43,4.31,8.15-1.44-2.72-2.87-5.43-4.31-8.15Z"/>+ </svg>+ }+ >+ </Card>+ <Card+ title="Haystack"+ href="/latest/integrations/haystack"+ iconType="duotone"+ icon={+ <svg version="1.1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 172 172">+ <path d="M0 0 C0.85369675 -0.0071553 1.70739349 -0.01431061 2.58695984 -0.02168274 C3.52052307 -0.02437164 4.4540863 -0.02706055 5.41593933 -0.02983093 C6.40160385 -0.03640213 7.38726837 -0.04297333 8.40280151 -0.04974365 C11.66914443 -0.0694768 14.93544139 -0.08113868 18.201828 -0.09111023 C19.87444985 -0.09722352 19.87444985 -0.09722352 21.58086205 -0.10346031 C27.48161509 -0.12432006 33.38234488 -0.13862281 39.28312683 -0.147995 C45.37974874 -0.15906554 51.47600627 -0.1934735 57.57249928 -0.23316669 C62.25910118 -0.25930038 66.94561579 -0.26768391 71.63228416 -0.27127647 C73.87975719 -0.27614998 76.12722613 -0.2877856 78.37462807 -0.30631447 C81.52020363 -0.33063851 84.66482993 -0.32975302 87.81047058 -0.32279968 C88.73796112 -0.33560974 89.66545166 -0.3484198 90.62104797 -0.36161804 C96.15238475 -0.31706859 99.50125664 0.3226312 103.80729675 4.14717102 C105.8682373 7.38111996 106.06190316 9.96075955 106.07472229 13.75762939 C106.08325226 14.9002272 106.09178223 16.04282501 106.10057068 17.220047 C106.09966431 18.46951111 106.09875793 19.71897522 106.0978241 21.00630188 C106.10454132 22.32534012 106.11125854 23.64437836 106.11817932 25.00338745 C106.13583849 28.61909204 106.14053461 32.23470196 106.14181089 35.85044551 C106.14323291 38.10988712 106.14750343 40.36930468 106.15280533 42.62874031 C106.17130416 50.51242221 106.17947598 58.3960365 106.17790222 66.27973938 C106.17670032 73.62683952 106.19781239 80.97364613 106.22938955 88.32067198 C106.25554299 94.63077136 106.26626994 100.94077481 106.2649914 107.25092798 C106.26448156 111.01876908 106.27017689 114.7863492 106.2913475 118.55413628 C106.31071492 122.09765824 106.3108652 125.64066983 106.29631424 129.18420982 C106.29322708 131.10242051 106.30991246 133.02063116 106.32719421 134.93876648 C106.31880524 136.08418411 106.31041626 137.22960175 106.30177307 138.409729 C106.30286261 139.40283553 106.30395214 140.39594206 106.30507469 141.41914272 C105.65918611 144.95887838 104.30608279 146.59510689 101.80729675 149.14717102 C98.98311617 150.99563202 96.80820199 151.40140558 93.46310425 151.42805481 C92.60106384 151.4394751 91.73902344 151.45089539 90.8508606 151.46266174 C89.44423912 151.46601532 89.44423912 151.46601532 88.00920105 151.46943665 C86.51826431 151.48433136 86.51826431 151.48433136 84.99720764 151.49952698 C81.70861543 151.52873962 78.42021615 151.54236865 75.1315155 151.55342102 C72.84471634 151.56492625 70.55791723 151.57644169 68.27111816 151.58796692 C63.47530723 151.60897802 58.67956326 151.62066921 53.88371277 151.62763977 C47.74327507 151.63860919 41.60372439 151.68666604 35.46356487 151.74358368 C30.73916043 151.780861 26.01496916 151.79124887 21.29043007 151.79431152 C19.02701275 151.80035996 16.76359898 151.81640177 14.50032616 151.84268188 C11.33072605 151.8768594 8.16328652 151.87303197 4.99357605 151.86006165 C4.06190109 151.87849121 3.13022614 151.89692078 2.1703186 151.91590881 C-3.61072638 151.84193862 -6.69669801 150.71658166 -10.92190838 146.72757721 C-13.41823999 143.62304946 -13.32147233 140.58321397 -13.33314514 136.77085876 C-13.33885529 135.65487808 -13.34456543 134.5388974 -13.35044861 133.38909912 C-13.35156647 132.16409195 -13.35268433 130.93908478 -13.35383606 129.67695618 C-13.35880096 128.38732468 -13.36376587 127.09769318 -13.36888123 125.76898193 C-13.38199725 122.22968105 -13.38844851 118.69039913 -13.39289188 115.15107775 C-13.39582362 112.93972424 -13.3999286 110.7283759 -13.40439606 108.51702499 C-13.41806828 101.59774888 -13.42773654 94.67848461 -13.43159795 87.75919592 C-13.43607956 79.77256996 -13.45363329 71.78611506 -13.48259854 63.79954112 C-13.5042134 57.62535235 -13.5143021 51.45121173 -13.51563907 45.27698553 C-13.51668715 41.5896512 -13.52259561 37.90247774 -13.54045868 34.21518326 C-13.5599973 30.1020748 -13.55565121 25.98932463 -13.54914856 21.87617493 C-13.55836334 20.65521622 -13.56757812 19.43425751 -13.57707214 18.17630005 C-13.57150299 17.05467972 -13.56593384 15.93305939 -13.56019592 14.77745056 C-13.56166248 13.80516825 -13.56312904 12.83288593 -13.56464005 11.83114052 C-12.95267206 7.41505849 -10.59231074 3.98017726 -7.19270325 1.14717102 C-4.45214497 0.23365159 -2.82279632 0.01626586 0 0 Z " fill="#04AF9D" transform="translate(49.19270324707031,25.852828979492188)"/>+ <path d="M0 0 C8.56257142 6.21206161 15.13410994 13.32451308 17 24 C17.27297659 28.82135157 17.23319759 33.63785146 17.1953125 38.46484375 C17.19157708 39.8673126 17.18873261 41.26978407 17.18673706 42.67225647 C17.17914221 46.34326527 17.15952998 50.01411295 17.1373291 53.68505859 C17.11678072 57.43910544 17.10771621 61.19317861 17.09765625 64.94726562 C17.07627965 72.29825011 17.04219457 79.64910238 17 87 C10.40625 87.265625 10.40625 87.265625 6.75 84.875 C0.66509373 78.51350708 0.58023841 72.90171474 0.62890625 64.50390625 C0.61152626 62.23111827 0.59394185 59.95833184 0.57617188 57.68554688 C0.56609811 54.1196813 0.5625041 50.55391633 0.56420898 46.98803711 C0.56063335 43.53469117 0.53327222 40.08211489 0.50390625 36.62890625 C0.51208878 35.56829483 0.5202713 34.50768341 0.52870178 33.41493225 C0.46495418 27.05536485 -0.1974533 23.16470457 -4 18 C-9.6759731 13.43454338 -15.44960295 11.44621744 -22.69140625 11.6171875 C-28.0049639 12.49828505 -31.80318766 15.70870248 -35.375 19.5625 C-38.1056093 23.65841395 -39.24839494 26.45184831 -39.4140625 31.421875 C-39.45273438 32.44667969 -39.49140625 33.47148438 -39.53125 34.52734375 C-39.57765625 36.12255859 -39.57765625 36.12255859 -39.625 37.75 C-39.66367188 38.82894531 -39.70234375 39.90789062 -39.7421875 41.01953125 C-39.83634248 43.67940932 -39.9192555 46.33968106 -40 49 C-35.71 49 -31.42 49 -27 49 C-27.04640625 47.74703125 -27.0928125 46.4940625 -27.140625 45.203125 C-27.17814813 43.55210719 -27.21454771 41.90106358 -27.25 40.25 C-27.28351563 39.425 -27.31703125 38.6 -27.3515625 37.75 C-27.45017991 31.63572044 -27.45017991 31.63572044 -25.19921875 28.45703125 C-21.9286803 26.29022691 -19.83483007 26.39450051 -16 27 C-12.7596102 29.5528916 -11.18636005 31.03812658 -10.39199829 35.13812256 C-10.40008514 36.3142511 -10.408172 37.49037964 -10.41650391 38.70214844 C-10.41416748 39.35476151 -10.41183105 40.00737457 -10.40942383 40.67976379 C-10.40781664 42.82379799 -10.44174967 44.96564853 -10.4765625 47.109375 C-10.48335483 48.60198185 -10.48807528 50.09459945 -10.49081421 51.58721924 C-10.50147242 54.70771633 -10.52812352 57.82749335 -10.56787109 60.94775391 C-10.6189141 64.95514049 -10.64391245 68.96211483 -10.65880775 72.96978378 C-10.67356324 76.78720505 -10.69880067 80.60452873 -10.7265625 84.421875 C-10.73392866 85.51933674 -10.73392866 85.51933674 -10.74144363 86.63896942 C-10.75765164 88.69789902 -10.78105095 90.75660427 -10.80712891 92.81542969 C-10.81998932 93.98837585 -10.83284973 95.16132202 -10.84609985 96.36981201 C-11 99 -11 99 -12 100 C-15.11063879 100.5184398 -17.17900563 100.5229155 -19.8828125 98.80078125 C-23.90413728 95.46089435 -26.67345258 92.72139517 -27.31884766 87.34277344 C-27.365979 85.86599121 -27.365979 85.86599121 -27.4140625 84.359375 C-27.45273438 83.28945312 -27.49140625 82.21953125 -27.53125 81.1171875 C-27.5621875 80.00601563 -27.593125 78.89484375 -27.625 77.75 C-27.66367188 76.62335938 -27.70234375 75.49671875 -27.7421875 74.3359375 C-27.83636983 71.55755891 -27.91927028 68.77880201 -28 66 C-31.63 66 -35.26 66 -39 66 C-39.02578125 66.79664062 -39.0515625 67.59328125 -39.078125 68.4140625 C-39.6679908 79.43382814 -39.6679908 79.43382814 -44 84 C-48.20403055 86.74175905 -51.06972187 87.12360029 -56 87 C-56.0931181 78.95858673 -56.16397484 70.91740273 -56.20724869 62.87558174 C-56.22802924 59.1404921 -56.25616923 55.4057742 -56.30175781 51.67089844 C-56.34556031 48.05930865 -56.36920859 44.448074 -56.37950897 40.83623886 C-56.38683928 39.46565036 -56.40115198 38.09507894 -56.42292023 36.72464371 C-56.607692 24.59815217 -54.42933664 15.03009114 -46 6 C-32.53468903 -6.63826692 -16.50134604 -8.5562535 0 0 Z " fill="#F8FCFC" transform="translate(115,54)"/>+ </svg>+ }+ >+ </Card></CardGroup>---diff --git a/docs/latest/integrations/llama-index.mdx b/docs/latest/integrations/llama-index.mdxnew file mode 100644index 00000000..023e7597--- /dev/null+++ b/docs/latest/integrations/llama-index.mdx@@ -0,0 +1,49 @@+---+title: 'Monitor LlamaIndex using OpenTelemetry'+sidebarTitle: 'LlamaIndex'+---++<Frame>+ <img src="/images/docs-llama-index-banner.gif" />+</Frame>++OpenLIT SDKs provide OpenTelemetry Auto-Instrumentation for monitoring LLM Application using LlamaIndex which includes performance and operation stats.++## Get Started++<Steps>+ <Step title="Install the OpenLIT Python SDK using pip">+ ```shell+ pip install openlit+ ```+ </Step>+ <Step title="Add the following two lines to your application code:">+ Integrating the OpenLIT into LLM applications is straightforward. Start monitoring for your LLM Application with just **one line of code**:++ ```python+ import openlit++ openlit.init(otlp_endpoint="otlp_backend_endpoint", otlp_headers="otlp_backend_token")+ ```++ Alternatively, you can configure the OTLP endpoint by setting the `OTEL_EXPORTER_OTLP_ENDPOINT` and the OTLP headers by setting the Appli`OTEL_EXPORTER_OTLP_HEADERS` environmentcation variable as recommended in the OpenTelemetry documentation.++ If your aim is primarily to test or develop without sending telemetry data to an external backend, you can initialize OpenLIT without any parameters. This approach directs all telemetry output to your console, enabling quick diagnostics and local debugging:++ ```python+ import openlit++ openlit.init()+ ```++ Refer to the OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.+ </Step>+</Steps>++---++<CardGroup cols={2}>+<Card title="Connections" href="/latest/connections/intro" icon='link'>+Connect to your existing Observablity Stack+</Card>+</CardGroup>\ No newline at end of filediff --git a/docs/mint.json b/docs/mint.jsonindex dea42e9a..8e05d4dc 100644--- a/docs/mint.json+++ b/docs/mint.json@@ -82,7 +82,9 @@"version": "latest","pages": ["latest/integrations/langchain",- "latest/integrations/litellm"+ "latest/integrations/litellm",+ "latest/integrations/llama-index",+ "latest/integrations/haystack"]}]diff --git a/sdk/python/src/openlit/__init__.py b/sdk/python/src/openlit/__init__.pyindex 339e627a..fc5838cc 100644--- a/sdk/python/src/openlit/__init__.py+++ b/sdk/python/src/openlit/__init__.py@@ -20,6 +20,8 @@ from openlit.instrumentation.mistral import MistralInstrumentorfrom openlit.instrumentation.bedrock import BedrockInstrumentorfrom openlit.instrumentation.vertexai import VertexAIInstrumentorfrom openlit.instrumentation.langchain import LangChainInstrumentor+from openlit.instrumentation.llamaindex import LlamaIndexInstrumentor+from openlit.instrumentation.haystack import HaystackInstrumentorfrom openlit.instrumentation.chroma import ChromaInstrumentorfrom openlit.instrumentation.pinecone import PineconeInstrumentorfrom openlit.instrumentation.transformers import TransformersInstrumentor@@ -150,6 +152,8 @@ def init(environment="default", application_name="default", tracer=None, otlp_en"bedrock": "boto3","vertexai": "vertexai","langchain": "langchain",+ "llama_index": "llama_index",+ "haystack": "haystack","chroma": "chromadb","pinecone": "pinecone","transformers": "transformers"@@ -198,6 +202,8 @@ def init(environment="default", application_name="default", tracer=None, otlp_en"bedrock": BedrockInstrumentor(),"vertexai": VertexAIInstrumentor(),"langchain": LangChainInstrumentor(),+ "llama_index": LlamaIndexInstrumentor(),+ "haystack": HaystackInstrumentor(),"chroma": ChromaInstrumentor(),"pinecone": PineconeInstrumentor(),"transformers": TransformersInstrumentor()diff --git a/sdk/python/src/openlit/instrumentation/anthropic/anthropic.py b/sdk/python/src/openlit/instrumentation/anthropic/anthropic.pyindex f5495186..af2f37a0 100644--- a/sdk/python/src/openlit/instrumentation/anthropic/anthropic.py+++ b/sdk/python/src/openlit/instrumentation/anthropic/anthropic.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring Anthropic API calls."""diff --git a/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.py b/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.pyindex 2b0c3e43..8fde34a7 100644--- a/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.py+++ b/sdk/python/src/openlit/instrumentation/anthropic/async_anthropic.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring Anthropic API calls."""diff --git a/sdk/python/src/openlit/instrumentation/chroma/chroma.py b/sdk/python/src/openlit/instrumentation/chroma/chroma.pyindex 206f8aca..3c89e5ce 100644--- a/sdk/python/src/openlit/instrumentation/chroma/chroma.py+++ b/sdk/python/src/openlit/instrumentation/chroma/chroma.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring ChromaDB."""diff --git a/sdk/python/src/openlit/instrumentation/cohere/cohere.py b/sdk/python/src/openlit/instrumentation/cohere/cohere.pyindex 52690adc..a2740da8 100644--- a/sdk/python/src/openlit/instrumentation/cohere/cohere.py+++ b/sdk/python/src/openlit/instrumentation/cohere/cohere.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring Cohere API calls."""diff --git a/sdk/python/src/openlit/instrumentation/haystack/__init__.py b/sdk/python/src/openlit/instrumentation/haystack/__init__.pynew file mode 100644index 00000000..35dccfbf--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/haystack/__init__.py@@ -0,0 +1,49 @@+# pylint: disable=useless-return, bad-staticmethod-argument, disable=duplicate-code+"""Initializer of Auto Instrumentation of Haystack Functions"""+from typing import Collection+import importlib.metadata+from opentelemetry.instrumentation.instrumentor import BaseInstrumentor+from wrapt import wrap_function_wrapper++from openlit.instrumentation.haystack.haystack import join_data++_instruments = ("haystack-ai >= 2.0.0",)++WRAPPED_METHODS = [+ {+ "package": "haystack.components.joiners.document_joiner",+ "object": "DocumentJoiner",+ "endpoint": "haystack.join_data",+ "wrapper": join_data,+ }+]++class HaystackInstrumentor(BaseInstrumentor):+ """An instrumentor for Cohere's client library."""++ def instrumentation_dependencies(self) -> Collection[str]:+ return _instruments++ def _instrument(self, **kwargs):+ application_name = kwargs.get("application_name")+ environment = kwargs.get("environment")+ tracer = kwargs.get("tracer")+ pricing_info = kwargs.get("pricing_info")+ trace_content = kwargs.get("trace_content")+ version = importlib.metadata.version("haystack-ai")++ for wrapped_method in WRAPPED_METHODS:+ wrap_package = wrapped_method.get("package")+ wrap_object = wrapped_method.get("object")+ gen_ai_endpoint = wrapped_method.get("endpoint")+ wrapper = wrapped_method.get("wrapper")+ wrap_function_wrapper(+ wrap_package,+ wrap_object,+ wrapper(gen_ai_endpoint, version, environment, application_name,+ tracer, pricing_info, trace_content),+ )++ @staticmethod+ def _uninstrument(self, **kwargs):+ passdiff --git a/sdk/python/src/openlit/instrumentation/haystack/haystack.py b/sdk/python/src/openlit/instrumentation/haystack/haystack.pynew file mode 100644index 00000000..e0479bb1--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/haystack/haystack.py@@ -0,0 +1,84 @@+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+"""+Module for monitoring Haystack applications.+"""++import logging+from opentelemetry.trace import SpanKind, Status, StatusCode+from opentelemetry.sdk.resources import TELEMETRY_SDK_NAME+from openlit.__helpers import handle_exception+from openlit.semcov import SemanticConvetion++# Initialize logger for logging potential issues and operations+logger = logging.getLogger(__name__)++def join_data(gen_ai_endpoint, version, environment, application_name,+ tracer, pricing_info, trace_content):+ """+ Creates a wrapper around a function call to trace and log its execution metrics.++ This function wraps any given function to measure its execution time,+ log its operation, and trace its execution using OpenTelemetry.++ Parameters:+ - gen_ai_endpoint (str): A descriptor or name for the endpoint being traced.+ - version (str): The version of the Haystack application.+ - environment (str): The deployment environment (e.g., 'production', 'development').+ - application_name (str): Name of the Haystack application.+ - tracer (opentelemetry.trace.Tracer): The tracer object used for OpenTelemetry tracing.+ - pricing_info (dict): Information about the pricing for internal metrics (currently not used).+ - trace_content (bool): Flag indicating whether to trace the content of the response.++ Returns:+ - function: A higher-order function that takes a function 'wrapped' and returns+ a new function that wraps 'wrapped' with additional tracing and logging.+ """++ def wrapper(wrapped, instance, args, kwargs):+ """+ An inner wrapper function that executes the wrapped function, measures execution+ time, and records trace data using OpenTelemetry.++ Parameters:+ - wrapped (Callable): The original function that this wrapper will execute.+ - instance (object): The instance to which the wrapped function belongs. This+ is used for instance methods. For static and classmethods,+ this may be None.+ - args (tuple): Positional arguments passed to the wrapped function.+ - kwargs (dict): Keyword arguments passed to the wrapped function.++ Returns:+ - The result of the wrapped function call.++ The wrapper initiates a span with the provided tracer, sets various attributes+ on the span based on the function's execution and response, and ensures+ errors are handled and logged appropriately.+ """+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:+ response = wrapped(*args, **kwargs)++ try:+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,+ SemanticConvetion.GEN_AI_SYSTEM_HAYSTACK)+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,+ gen_ai_endpoint)+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,+ environment)+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,+ SemanticConvetion.GEN_AI_TYPE_FRAMEWORK)+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,+ application_name)+ span.set_status(Status(StatusCode.OK))++ # Return original response+ return response++ except Exception as e:+ handle_exception(span, e)+ logger.error("Error in trace creation: %s", e)++ # Return original response+ return response++ return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/langchain/langchain.py b/sdk/python/src/openlit/instrumentation/langchain/langchain.pyindex 64148b96..998c2fd4 100644--- a/sdk/python/src/openlit/instrumentation/langchain/langchain.py+++ b/sdk/python/src/openlit/instrumentation/langchain/langchain.py@@ -60,7 +60,7 @@ def general_wrap(gen_ai_endpoint, version, environment, application_name,try:span.set_attribute(TELEMETRY_SDK_NAME, "openlit")span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,- "langchain")+ SemanticConvetion.GEN_AI_SYSTEM_LANGCHAIN)span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,gen_ai_endpoint)span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,diff --git a/sdk/python/src/openlit/instrumentation/llamaindex/__init__.py b/sdk/python/src/openlit/instrumentation/llamaindex/__init__.pynew file mode 100644index 00000000..ab315ee1--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/llamaindex/__init__.py@@ -0,0 +1,55 @@+# pylint: disable=useless-return, bad-staticmethod-argument, disable=duplicate-code+"""Initializer of Auto Instrumentation of LlamaIndex Functions"""+from typing import Collection+import importlib.metadata+from opentelemetry.instrumentation.instrumentor import BaseInstrumentor+from wrapt import wrap_function_wrapper++from openlit.instrumentation.llamaindex.llamaindex import load_data++_instruments = ("llama-index >= 0.10.0",)++WRAPPED_METHODS = [+ {+ "package": "llama_index.core.readers",+ "object": "SimpleDirectoryReader.load_data",+ "endpoint": "llamaindex.load_data",+ "wrapper": load_data,+ },+ {+ "package": "llama_index.core.node_parser",+ "object": "SentenceSplitter.get_nodes_from_documents",+ "endpoint": "llamaindex.data_splitter",+ "wrapper": load_data,+ },+]++class LlamaIndexInstrumentor(BaseInstrumentor):+ """An instrumentor for Cohere's client library."""++ def instrumentation_dependencies(self) -> Collection[str]:+ return _instruments++ def _instrument(self, **kwargs):+ application_name = kwargs.get("application_name")+ environment = kwargs.get("environment")+ tracer = kwargs.get("tracer")+ pricing_info = kwargs.get("pricing_info")+ trace_content = kwargs.get("trace_content")+ version = importlib.metadata.version("llama-index")++ for wrapped_method in WRAPPED_METHODS:+ wrap_package = wrapped_method.get("package")+ wrap_object = wrapped_method.get("object")+ gen_ai_endpoint = wrapped_method.get("endpoint")+ wrapper = wrapped_method.get("wrapper")+ wrap_function_wrapper(+ wrap_package,+ wrap_object,+ wrapper(gen_ai_endpoint, version, environment, application_name,+ tracer, pricing_info, trace_content),+ )++ @staticmethod+ def _uninstrument(self, **kwargs):+ passdiff --git a/sdk/python/src/openlit/instrumentation/llamaindex/llamaindex.py b/sdk/python/src/openlit/instrumentation/llamaindex/llamaindex.pynew file mode 100644index 00000000..40e0e000--- /dev/null+++ b/sdk/python/src/openlit/instrumentation/llamaindex/llamaindex.py@@ -0,0 +1,86 @@+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+"""+Module for monitoring LlamaIndex applications.+"""++import logging+from opentelemetry.trace import SpanKind, Status, StatusCode+from opentelemetry.sdk.resources import TELEMETRY_SDK_NAME+from openlit.__helpers import handle_exception+from openlit.semcov import SemanticConvetion++# Initialize logger for logging potential issues and operations+logger = logging.getLogger(__name__)++def load_data(gen_ai_endpoint, version, environment, application_name,+ tracer, pricing_info, trace_content):+ """+ Creates a wrapper around a function call to trace and log its execution metrics.++ This function wraps any given function to measure its execution time,+ log its operation, and trace its execution using OpenTelemetry.++ Parameters:+ - gen_ai_endpoint (str): A descriptor or name for the endpoint being traced.+ - version (str): The version of the LlamaIndex application.+ - environment (str): The deployment environment (e.g., 'production', 'development').+ - application_name (str): Name of the LlamaIndex application.+ - tracer (opentelemetry.trace.Tracer): The tracer object used for OpenTelemetry tracing.+ - pricing_info (dict): Information about the pricing for internal metrics (currently not used).+ - trace_content (bool): Flag indicating whether to trace the content of the response.++ Returns:+ - function: A higher-order function that takes a function 'wrapped' and returns+ a new function that wraps 'wrapped' with additional tracing and logging.+ """++ def wrapper(wrapped, instance, args, kwargs):+ """+ An inner wrapper function that executes the wrapped function, measures execution+ time, and records trace data using OpenTelemetry.++ Parameters:+ - wrapped (Callable): The original function that this wrapper will execute.+ - instance (object): The instance to which the wrapped function belongs. This+ is used for instance methods. For static and classmethods,+ this may be None.+ - args (tuple): Positional arguments passed to the wrapped function.+ - kwargs (dict): Keyword arguments passed to the wrapped function.++ Returns:+ - The result of the wrapped function call.++ The wrapper initiates a span with the provided tracer, sets various attributes+ on the span based on the function's execution and response, and ensures+ errors are handled and logged appropriately.+ """+ with tracer.start_as_current_span(gen_ai_endpoint, kind= SpanKind.CLIENT) as span:+ response = wrapped(*args, **kwargs)++ try:+ span.set_attribute(TELEMETRY_SDK_NAME, "openlit")+ span.set_attribute(SemanticConvetion.GEN_AI_SYSTEM,+ SemanticConvetion.GEN_AI_SYSTEM_LLAMAINDEX)+ span.set_attribute(SemanticConvetion.GEN_AI_ENDPOINT,+ gen_ai_endpoint)+ span.set_attribute(SemanticConvetion.GEN_AI_ENVIRONMENT,+ environment)+ span.set_attribute(SemanticConvetion.GEN_AI_TYPE,+ SemanticConvetion.GEN_AI_TYPE_FRAMEWORK)+ span.set_attribute(SemanticConvetion.GEN_AI_APPLICATION_NAME,+ application_name)+ span.set_attribute(SemanticConvetion.GEN_AI_RETRIEVAL_SOURCE,+ response[0].metadata["file_path"])+ span.set_status(Status(StatusCode.OK))++ # Return original response+ return response++ except Exception as e:+ handle_exception(span, e)+ logger.error("Error in trace creation: %s", e)++ # Return original response+ return response++ return wrapperdiff --git a/sdk/python/src/openlit/instrumentation/mistral/async_mistral.py b/sdk/python/src/openlit/instrumentation/mistral/async_mistral.pyindex f0b09054..f35d6e78 100644--- a/sdk/python/src/openlit/instrumentation/mistral/async_mistral.py+++ b/sdk/python/src/openlit/instrumentation/mistral/async_mistral.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring Mistral API calls."""diff --git a/sdk/python/src/openlit/instrumentation/mistral/mistral.py b/sdk/python/src/openlit/instrumentation/mistral/mistral.pyindex 86f09931..cdfd430d 100644--- a/sdk/python/src/openlit/instrumentation/mistral/mistral.py+++ b/sdk/python/src/openlit/instrumentation/mistral/mistral.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring Mistral API calls."""diff --git a/sdk/python/src/openlit/instrumentation/pinecone/pinecone.py b/sdk/python/src/openlit/instrumentation/pinecone/pinecone.pyindex bbd78248..47e41511 100644--- a/sdk/python/src/openlit/instrumentation/pinecone/pinecone.py+++ b/sdk/python/src/openlit/instrumentation/pinecone/pinecone.py@@ -1,4 +1,4 @@-# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument+# pylint: disable=duplicate-code, broad-exception-caught, too-many-statements, unused-argument, possibly-used-before-assignment"""Module for monitoring Pinecone."""diff --git a/sdk/python/src/openlit/semcov/__init__.py b/sdk/python/src/openlit/semcov/__init__.pyindex fcada45b..1c2dfc9b 100644--- a/sdk/python/src/openlit/semcov/__init__.py+++ b/sdk/python/src/openlit/semcov/__init__.py@@ -88,6 +88,8 @@ class SemanticConvetion:GEN_AI_SYSTEM_BEDROCK = "bedrock"GEN_AI_SYSTEM_VERTEXAI = "vertexai"GEN_AI_SYSTEM_LANGCHAIN = "langchain"+ GEN_AI_SYSTEM_LLAMAINDEX = "llama_index"+ GEN_AI_SYSTEM_HAYSTACK = "haystack"# Vector DBDB_REQUESTS = "db.total.requests"