Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
TypeScript agent-harnessagent-harness-engineeringai-engineeringai-observabilityamd-gpuclaude-codeclickhousecodexcoding-agentcoding-agentscursorgenaigpu-monitoringllm-observabilityllmopsnvidia-smiobservabilityopen-sourceopenaiopentelemetry
Update README
patcher9605b10ef14ef
diff --git a/README.md b/README.mdindex c6862a70..7ba6a030 100644--- a/README.md+++ b/README.md@@ -1,9 +1,10 @@<div align="center">-<img src="https://github.com/dokulabs/.github/blob/main/profile/assets/favicon.png?raw=true" alt="Doku Logo" width="10%"><h1>Doku: Open Source Observability for LLMs</h1>+<img src="https://github.com/dokulabs/.github/blob/main/profile/assets/favicon.png?raw=true" alt="Doku Logo" width="10%"><h1>OpenLIT:+OpenTelemetry-native LLM Application Observability</h1>-**[Documentation](https://docs.dokulabs.com/) | [Quickstart](#-getting-started-with-doku) | [Python SDK](https://github.com/dokulabs/dokumetry-python) | [Node SDK](https://github.com/dokulabs/dokumetry-node) | [Helm Chart](https://github.com/dokulabs/helm)**+<!-- **[Documentation](https://docs.dokulabs.com/) | [Quickstart](#-getting-started-with-doku) | [Python SDK](https://github.com/dokulabs/dokumetry-python) | [Node SDK](https://github.com/dokulabs/dokumetry-node) | [Helm Chart](https://github.com/dokulabs/helm)** -->-[](https://github.com/dokulabs/doku)+<!-- [](https://github.com/dokulabs/doku)[](https://github.com/dokulabs/doku/blob/main/LICENSE)[](https://pepy.tech/project/dokumetry)[](https://github.com/dokulabs/doku/pulse)@@ -15,274 +16,71 @@[](https://github.com/dokulabs/doku/tags)[](https://github.com/dokulabs/helm/tags)[](https://pypi.org/project/dokumetry/)-[](https://www.npmjs.com/package/dokumetry)+[](https://www.npmjs.com/package/dokumetry) --></div>-Doku is an **open-source LLMOps tool** engineered to enables developers with comprehensive capabilities to monitor, analyze, and optimize LLM applications. It provides valuable real-time data on **LLM usage, performance, and costs**. Through seamless integrations with leading LLM platforms, including OpenAI, Cohere, and Anthropic, Doku acts as a central command center for all your LLM needs. It effectively guides your efforts, ensuring that your LLM applications not only operate at peak efficiency but also scale successfully.+OpenLIT is an **OpenTelemetry-native** GenAI and LLM Application Observability tool. It's designed to make the integration process of observability into GenAI projects as easy as pie – literally, with just **a single line of code**. Whether you're working with popular LLM Libraries such as OpenAI and HuggingFace or leveraging vector databases like ChromaDB, OpenLIT ensures your applications are monitored seamlessly, providing critical insights to improve performance and reliability.-https://github.com/dokulabs/doku/assets/150420377/80281574-40c0-404a-a0bc-bdc5c631c9d8+This project proudly follows the [Semantic Conventions](https://github.com/open-telemetry/semantic-conventions/tree/main/docs/gen-ai) of the OpenTelemetry community, consistently updating to align with the latest standards in observability.-## Why use Doku?-Get advanced monitoring and evaluation for your LLM applications with these key benefits:+## What is LIT?+LIT stands for Learning Interpretability Tool. It refers to a visual, interactive model-understanding and data visualization tool ad a term introduced by [Google](https://developers.google.com/machine-learning/glossary#learning-interpretability-tool-lit).+## Features+- **OpenTelemetry-native**: Native support ensures that integrating OpenLIT into your projects feels more like a natural extension rather than an additional layer of complexity.- **Granular Usage Insights of your LLM Applications**: Assess your LLM's performance and costs with fine-grained control, breaking down metrics by environment (such as staging or production) or application, to optimize for efficiency and scalability.-- **Real-Time Data Streaming**: Unlike other platforms where you might wait minutes to see your data due to data being sent in batches, Doku is able to display data as it streams. This immediate insight enables quick decision-making and adjustments.-- **Zero Added Latency**: Doku's smart data handling ensures rapid data processing without impacting your application's performance, maintaining the responsiveness of your LLM applications.-- **Connect to Observability Platforms**: Doku seamlessly connects with leading observability platforms like Grafana Cloud and Datadog, among others to automatically export data.+- **Vendor-Neutral SDKs**: In the spirit of OpenTelemetry, OpenLIT's SDKs are agnostic of the backend vendors. This means you can confidently use OpenLIT with various telemetry backends, like Grafana Tempo, without worrying about compatibility issues.-## How it works?-+## 🚀 Getting Started-### Step 1: Instrument your Application-Integrating the `dokumetry` SDK into LLM applications is straightforward with SDKs designed for Python and NodeJS. Start monitoring for your LLM Application with just **two lines of code**:+## Step 1: Install OpenLIT SDK-For Python-```python python-import dokumetry--dokumetry.init(llm=openai, doku_url="YOUR_DOKU_INGESTER_URL", api_key="YOUR_DOKU_TOKEN")-```-For NodeJS--```javascript-import DokuMetry from 'dokumetry';--DokuMetry.init({llm: openai, dokuUrl: "YOUR_DOKU_INGESTER_URL", apiKey: "YOUR_DOKU_TOKEN"})-```--### Step 2: Data processed by Doku Ingester-Once the `dokumetry` SDKs are configured in your LLM application, Monitoring data starts streaming to the [Doku Ingester](https://github.com/dokulabs/doku/tree/main/src/ingester#readme). It processes and safely stores your data in ClickHouse, keeping your LLM Monitoring data **secure** and **compliant** in your environment.--You can choose to use a new ClickHouse database setup or connect to your existing one to work with Doku.---### Step 3: Visualize and analyze--With your LLM monitoring data processed and securely stored, you can now leverage the Doku UI for in-depth visualization and analysis. Doku UI allows you to explore LLM costs, token usage, performance metrics, and user interactions in an intuitive interface. This powerful tool enhances your ability to observe and optimize your LLM applications, ensuring you make data-driven decisions for improvement.--For those with a preferred observability platform, you can also integrate and visualize this data elsewhere with ease. This flexibility ensures optimal monitoring workflow integration, regardless of your platform choice. For more details on how to set up these connections, check out the [Connections](https://docs.dokulabs.com/latest/connections/intro) guide.--## 🚀 Getting Started with Doku--### 💿 Installation-Jumpstart your journey with Doku by deploying it via our Helm chart, designed to simplify the installation process on any Kubernetes cluster.--#### Docker-To install the Doku using Docker, follow these steps:--1. Create `docker-compose.yml`- ```yaml- version: '3.8'-- services:- clickhouse:- image: clickhouse/clickhouse-server:24.2.2- container_name: clickhouse- environment:- CLICKHOUSE_PASSWORD: ${DOKU_DB_PASSWORD:-DOKU}- CLICKHOUSE_USER: ${DOKU_DB_USER:-default}- volumes:- - clickhouse-data:/var/lib/clickhouse- ports:- - "9000:9000"- - "8123:8123"- restart: always-- doku-ingester:- image: ghcr.io/dokulabs/doku-ingester:latest- container_name: doku-ingester- environment:- DOKU_DB_HOST: clickhouse- DOKU_DB_PORT: 9000- DOKU_DB_NAME: ${DOKU_DB_NAME:-default}- DOKU_DB_USER: ${DOKU_DB_USER:-default}- DOKU_DB_PASSWORD: ${DOKU_DB_PASSWORD:-DOKU}- ports:- - "9044:9044"- depends_on:- - clickhouse- restart: always-- doku-client:- image: ghcr.io/dokulabs/doku-client:latest- container_name: doku-client- environment:- INIT_DB_HOST: clickhouse- INIT_DB_PORT: 8123- INIT_DB_DATABASE: ${DOKU_DB_NAME:-default}- INIT_DB_USERNAME: ${DOKU_DB_USER:-default}- INIT_DB_PASSWORD: ${DOKU_DB_PASSWORD:-DOKU}- SQLITE_DATABASE_URL: file:/app/client/data/data.db- ports:- - "3000:3000"- depends_on:- - clickhouse- volumes:- - doku-client-data:/app/client/data- restart: always-- volumes:- clickhouse-data:- doku-client-data:- ```--2. Start Docker Compose- ```shell- docker-compose up -d- ```--#### Kubernetes--To install the Doku Helm chart, follow these steps:--1. Add the Doku Helm repository to your Helm setup:-- ```shell- helm repo add dokulabs https://dokulabs.github.io/helm/- ```--2. Update your Helm repositories to fetch the latest chart information:-- ```shell- helm repo update- ```--3. Install the Doku chart with the release name `doku`:-- ```shell- helm install doku dokulabs/doku- ```--For a detailed list of configurable parameters for the Helm chart, refer to the `values.yaml` file in the [Helm chart](https://github.com/dokulabs/doku/tree/main/helm/doku).--### Other Installation Methods--[](https://railway.app/template/tZQJlB?referralCode=8Gb9bk)--### 🔑 Access Doku UI and Generate an API Key--With Doku running, the next step is to access the Doku UI and generate an API key for secure communication between your applications and Doku.--1. Open your browser and go to Doku UI at `127.0.0.1:3000/login`-2. Login using theb default credentials- - Email as `[email protected]`- - Password as `dokulabsuser`-3. Once you have logged into Doku UI, Go to [API Keys page](http://127.0.0.1:3000/api-keys) and Create an API Key. Copy the generated API Key.--> 💡 **Tip:** Alternatively, you can use the HTTP API to create your Doku API Key. For further details, take a look at the [API Reference](https://docs.dokulabs.com/latest/api-reference/endpoint/api-keys/create) section.--### ⚡️ Instrument your Application--Choose the appropriate SDK for your LLM application's programming language and follow the steps to integrate monitoring with just **two lines of code**.--#### Python--Install the `dokumetry` [Python SDK](https://pypi.org/project/dokumetry/) using pip:--```shell-pip install dokumetry-```--Add the following two lines to your application code:--```python-import dokumetry--dokumetry.init(llm=client, doku_url="YOUR_DOKU_INGESTER_URL", api_key="YOUR_DOKU_TOKEN")-```--##### Example Usage for monitoring `OpenAI` Usage:--```python-from openai import OpenAI-import dokumetry--client = OpenAI(- api_key="YOUR_OPENAI_KEY"-)--# Pass the above `client` object along with your Doku Ingester URL and API key and this will make sure that all OpenAI calls are automatically tracked.-dokumetry.init(llm=client, doku_url="YOUR_DOKU_INGESTER_URL", api_key="YOUR_DOKU_TOKEN")--chat_completion = client.chat.completions.create(- messages=[- {- "role": "user",- "content": "What is LLM Observability",- }- ],- model="gpt-3.5-turbo",-)-```--Refer to the `dokumetry` [Python SDK repository](https://github.com/dokulabs/dokumetry-python) for more advanced configurations and use cases.--#### Node--Install the `dokumetry` [NodeJS SDK](https://www.npmjs.com/package/dokumetry) using npm:--```shell-npm install dokumetry+```bash+pip install openlit```-Add the following two lines to your application code:+### Step 2: Instrument your Application+Integrating the OpenLIT into LLM applications is straightforward. Start monitoring for your LLM Application with just **one line of code**:-```javascript-import DokuMetry from 'dokumetry';+For Python+```python python+import openlit-DokuMetry.init({llm: openai, dokuUrl: "YOUR_DOKU_INGESTER_URL", apiKey: "YOUR_DOKU_TOKEN"})+openlit.init()```-##### Example Usage for monitoring `OpenAI` Usage:--```javascript-import OpenAI from 'openai';-import DokuMetry from 'dokumetry';--const openai = new OpenAI({- apiKey: 'My API Key', // defaults to process.env["OPENAI_API_KEY"]-});+This will Auto instrument you code for collecting LLM Observability data.-// Pass the above `openai` object along with your Doku Ingester URL and API key and this will make sure that all OpenAI calls are automatically tracked.-DokuMetry.init({llm: openai, dokuUrl: "YOUR_DOKU_INGESTER_URL", apiKey: "YOUR_DOKU_TOKEN"})+### Step 2: Send to OTLP Backend-async function main() {- const chatCompletion = await openai.chat.completions.create({- messages: [{ role: 'user', content: 'What are the key to effective observability?' }],- model: 'gpt-3.5-turbo',- });-}+By default, OpenLIT directs traces and metrics straight to your console. To forward telemetry data to an HTTP OTLP endpoint, such as the OpenTelemetry Collector, set the `otlp_endpoint` parameter with the desired endpoint. Alternatively, you can configure the endpoint by setting the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable as recommended in the OpenTelemetry documentation.-main();-```--Refer to the `dokumetry` [NodeJS SDK repository](https://github.com/dokulabs/dokumetry-node) for more advanced configurations and use cases.+To send telemetry to OpenTelemetry backends requiring authentication, set the `otlp_headers` parameter with its desired value. Alternatively, you can configure the endpoint by setting the `OTEL_EXPORTER_OTLP_HEADERS` environment variable as recommended in the OpenTelemetry documentation.-### Visualize and Analyze+## 🌱 Contributing-Once you have Doku Ingester and `DokuMetry` SDKs set up, you can instantly get insights into how your LLM applications in the Doku Client UI. Just head over to `127.0.0.1:3000` on your browser to start exploring.+Whether it's big or small, we love contributions 💚. Check out our [Contribution guide](../../CONTRIBUTING.md) to get started-+Unsure where to start? Here are a few ways to get involved:-With Doku, you get a simple, powerful view into important info like how much you’re spending on LLMs, which parts of your app are using them the most, and how well they’re performing. Find out which LLM models are favorites among your applications, and dive deep into performance details to make smart decisions. This setup is perfect for optimizing your app performance and keeping an eye on costs.+- Join our [Slack channel](https://join.slack.com/t/openlit/shared_invite/zt-2etnfttwg-TjP_7BZXfYg84oAukY8QRQ) to discuss ideas, share feedback, and connect with both our team and the wider OpenLIT community.-## Security+Your input helps us grow and improve, and we're here to support you every step of the way.-Doku uses key based authentication mechanism to ensure the security of your data and as Doku is self-hosted, The data stays within your environment.+## 💚 Community & Support-## Contributing+Connect with the OpenLIT community and maintainers for support, discussions, and updates:-We welcome contributions to the Doku project. Please refer to [CONTRIBUTING](CONTRIBUTING) for detailed guidelines on how you can participate.+- 🌟 If you like it, Leave a star on our [GitHub](https://github.com/open-lit/openlit/)+- 🌍 Join our [Slack](https://join.slack.com/t/openlit/shared_invite/zt-2etnfttwg-TjP_7BZXfYg84oAukY8QRQ) Community for live interactions and questions.+- 🐞 Report bugs on our [GitHub Issues](https://github.com/open-lit/openlit/issues) to help us improve OpenLIT.+- 𝕏 Follow us on [X](https://twitter.com/openlit) for the latest updates and news.## License-Doku is available under the [Apache-2.0 license](LICENSE).--## Support--For support, issues, or feature requests, submit an issue through the [GitHub issues](https://github.com/dokulabs/doku/issues) associated with this repository.+OpenLIT is available under the [Apache-2.0 license](LICENSE).## Visualize! Analyze! Optimize!-Join us on this voyage to reshape the future of AI Observability. Share your thoughts, suggest features, and explore contributions. Engage with us on [GitHub](https://github.com/dokulabs/doku) and be part of Doku's community-led innovation.+Join us on this voyage to reshape the future of AI Observability. Share your thoughts, suggest features, and explore contributions. Engage with us on [GitHub](https://github.com/open-lit/openlit) and be part of OpenLIT's community-led innovation.