Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
Self-host: precise local setup docs + CLICKHOUSE_SECURE support (local ClickHouse) [Devin] (#1242)
devin-ai-integration[bot]e55818377373
diff --git a/app/README.md b/app/README.mdindex 517c7815..4670d040 100644--- a/app/README.md+++ b/app/README.md@@ -1,3 +1,61 @@+Restart local stack and verify++- Restart services:+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml down --remove-orphans+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml up -d++- Check logs:+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml logs --since=90s api+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml logs --since=90s dashboard+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml logs --since=90s otelcollector+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml logs --since=90s clickhouse++- Open dashboard:+ http://localhost:3000/signin++- Generate a trace (example):+ AGENTOPS_API_KEY=<key> \+ AGENTOPS_API_ENDPOINT=http://localhost:8000 \+ AGENTOPS_APP_URL=http://localhost:3000 \+ AGENTOPS_EXPORTER_ENDPOINT=http://localhost:4318/v1/traces \+ OPENAI_API_KEY=<openai_key> \+ python examples/openai/openai_example_sync.py++- Verify ClickHouse and dashboard:+ curl -s -u default:password "http://localhost:8123/?query=SELECT%20count()%20FROM%20otel_2.otel_traces%20WHERE%20TraceId%20=%20'<TRACE_ID>'"++ http://localhost:3000/traces?trace_id=<TRACE_ID>+++Local ClickHouse (self-hosted)+- Set in .env:+ CLICKHOUSE_HOST=127.0.0.1+ CLICKHOUSE_PORT=8123+ CLICKHOUSE_USER=default+ CLICKHOUSE_PASSWORD=password+ CLICKHOUSE_DATABASE=otel_2+ CLICKHOUSE_SECURE=false+ CLICKHOUSE_ENDPOINT=http://clickhouse:8123+ CLICKHOUSE_USERNAME=default++- Start services (includes otelcollector + local ClickHouse):+ docker compose -f compose.yaml -f opentelemetry-collector/compose.yaml up -d++- Initialize ClickHouse schema:+ curl -u default:password 'http://localhost:8123/?query=CREATE%20DATABASE%20IF%20NOT%20EXISTS%20otel_2'+ curl --data-binary @app/clickhouse/migrations/0000_init.sql -u default:password 'http://localhost:8123/?query='++- Run example with local OTLP exporter:+ AGENTOPS_API_KEY=<your_key> \+ AGENTOPS_API_ENDPOINT=http://localhost:8000 \+ AGENTOPS_APP_URL=http://localhost:3000 \+ AGENTOPS_EXPORTER_ENDPOINT=http://localhost:4318/v1/traces \+ OPENAI_API_KEY=<openai_key> \+ python examples/openai/openai_example_sync.py++- Verify:+ - Dashboard: http://localhost:3000/traces?trace_id=<printed_id>+ - CH rows: curl -s -u default:password "http://localhost:8123/?query=SELECT%20count()%20FROM%20otel_2.otel_traces%20WHERE%20TraceId%20=%20'<TRACE_ID>'"# AgentOps[](https://www.elastic.co/licensing/elastic-license)@@ -312,6 +370,7 @@ AgentOps requires several external services. Here's how to set them up:```4. Run migrations: `supabase db push`+For Linux environments with CLI install issues, see docs/local_supabase_linux.md for a manual binary install and env mapping steps.**Option B: Cloud Supabase**1. Create a new project at [supabase.com](https://supabase.com)diff --git a/app/api/agentops/api/db/clickhouse_client.py b/app/api/agentops/api/db/clickhouse_client.pyindex 359a76e3..9064ec11 100644--- a/app/api/agentops/api/db/clickhouse_client.py+++ b/app/api/agentops/api/db/clickhouse_client.py@@ -13,6 +13,7 @@ from agentops.api.environment import (CLICKHOUSE_PASSWORD,CLICKHOUSE_PORT,CLICKHOUSE_USER,+ CLICKHOUSE_SECURE,)# Global variables to store client instances@@ -37,7 +38,7 @@ class ConnectionConfig:database: str = CLICKHOUSE_DATABASEusername: str = CLICKHOUSE_USERpassword: str = CLICKHOUSE_PASSWORD- secure: bool = True+ secure: bool = CLICKHOUSE_SECUREdef __init__(self) -> None:"""Non-instantiable class has a lower chance of being printed."""diff --git a/app/api/agentops/api/environment.py b/app/api/agentops/api/environment.pyindex 0831a4c6..9a8c6ced 100644--- a/app/api/agentops/api/environment.py+++ b/app/api/agentops/api/environment.py@@ -18,6 +18,7 @@ CLICKHOUSE_PORT: int = int(os.getenv("CLICKHOUSE_PORT", 0))CLICKHOUSE_USER: str = os.getenv("CLICKHOUSE_USER", "")CLICKHOUSE_PASSWORD: str = os.getenv("CLICKHOUSE_PASSWORD", "")CLICKHOUSE_DATABASE: str = os.getenv("CLICKHOUSE_DATABASE", "")+CLICKHOUSE_SECURE: bool = os.getenv("CLICKHOUSE_SECURE", "false").lower() in ("1", "true", "yes")PROFILING_ENABLED: bool = os.environ.get("PROFILING_ENABLED", "false").lower() == "true"diff --git a/app/compose.yaml b/app/compose.yamlindex a9dd8609..ef5a130c 100644--- a/app/compose.yaml+++ b/app/compose.yaml@@ -37,6 +37,10 @@ services:CLICKHOUSE_SECURE: ${CLICKHOUSE_SECURE}CLICKHOUSE_ENDPOINT: ${CLICKHOUSE_ENDPOINT}CLICKHOUSE_USERNAME: ${CLICKHOUSE_USERNAME}+ SUPABASE_S3_BUCKET: ${SUPABASE_S3_BUCKET}+ SUPABASE_S3_LOGS_BUCKET: ${SUPABASE_S3_LOGS_BUCKET}+ SUPABASE_S3_ACCESS_KEY_ID: ${SUPABASE_S3_ACCESS_KEY_ID}+ SUPABASE_S3_SECRET_ACCESS_KEY: ${SUPABASE_S3_SECRET_ACCESS_KEY}network_mode: 'host'volumes:- ./api:/app/apidiff --git a/app/opentelemetry-collector/compose.yaml b/app/opentelemetry-collector/compose.yamlindex af3e0e2a..b0c6e23e 100644--- a/app/opentelemetry-collector/compose.yaml+++ b/app/opentelemetry-collector/compose.yaml@@ -25,7 +25,7 @@ services:- TRACES_TABLE_NAME=${TRACES_TABLE_NAME:-otel_traces}- CLICKHOUSE_TTL=${CLICKHOUSE_TTL:-12h}- CLICKHOUSE_TIMEOUT=${CLICKHOUSE_TIMEOUT:-10s}- - JWT_SECRET=${JWT_SECRET:-}+ - JWT_SECRET=${JWT_SECRET_KEY:-super-secret-jwt-token-with-at-least-32-characters-long}clickhouse:image: clickhouse/clickhouse-server:24.12diff --git a/docs/local_clickhouse_setup.md b/docs/local_clickhouse_setup.mdnew file mode 100644index 00000000..e467b302--- /dev/null+++ b/docs/local_clickhouse_setup.md@@ -0,0 +1,43 @@+Restart and verify++docker compose -f app/compose.yaml -f app/opentelemetry-collector/compose.yaml down --remove-orphans+docker compose -f app/compose.yaml -f app/opentelemetry-collector/compose.yaml up -d+docker compose -f app/compose.yaml -f app/opentelemetry-collector/compose.yaml logs --since=90s api+docker compose -f app/compose.yaml -f app/opentelemetry-collector/compose.yaml logs --since=90s otelcollector+docker compose -f app/compose.yaml -f app/opentelemetry-collector/compose.yaml logs --since=90s clickhouse++Open http://localhost:3000/signin+++Local ClickHouse Setup++1) Environment (.env)+CLICKHOUSE_HOST=127.0.0.1+CLICKHOUSE_PORT=8123+CLICKHOUSE_USER=default+CLICKHOUSE_PASSWORD=password+CLICKHOUSE_DATABASE=otel_2+CLICKHOUSE_SECURE=false+CLICKHOUSE_ENDPOINT=http://clickhouse:8123+CLICKHOUSE_USERNAME=default++2) Start services (includes local ClickHouse and otelcollector)+docker compose -f app/compose.yaml -f app/opentelemetry-collector/compose.yaml up -d++3) Initialize ClickHouse schema+curl -u default:password 'http://localhost:8123/?query=CREATE%20DATABASE%20IF%20NOT%20EXISTS%20otel_2'+curl --data-binary @app/clickhouse/migrations/0000_init.sql -u default:password 'http://localhost:8123/?query='++4) Run OpenAI example with local OTLP+AGENTOPS_API_KEY=<agentops_key> \+AGENTOPS_API_ENDPOINT=http://localhost:8000 \+AGENTOPS_APP_URL=http://localhost:3000 \+AGENTOPS_EXPORTER_ENDPOINT=http://localhost:4318/v1/traces \+OPENAI_API_KEY=<openai_key> \+python examples/openai/openai_example_sync.py++5) Verify+- Dashboard: http://localhost:3000/traces?trace_id=<TRACE_ID>+- ClickHouse:+ curl -s -u default:password "http://localhost:8123/?query=SHOW%20TABLES%20FROM%20otel_2"+ curl -s -u default:password "http://localhost:8123/?query=SELECT%20count()%20FROM%20otel_2.otel_traces%20WHERE%20TraceId%20=%20'<TRACE_ID>'"diff --git a/docs/local_supabase_linux.md b/docs/local_supabase_linux.mdnew file mode 100644index 00000000..c23c838c--- /dev/null+++ b/docs/local_supabase_linux.md@@ -0,0 +1,56 @@+Local Supabase on Linux (CLI install fallback)++1) Install CLI+- Download latest Linux x86_64 binary from https://github.com/supabase/cli/releases/latest+- Extract to ~/.supabase/bin and chmod +x+- Add to PATH:+ export PATH="$HOME/.supabase/bin:$PATH"+- Verify:+ supabase --version++2) Initialize and start in app/+- cd ~/repos/agentops/app+- supabase init+- supabase start++3) Capture credentials from output+- URL: http://127.0.0.1:54321+- anon key+- service_role key+- Postgres: host 127.0.0.1, port 54322, user postgres, password postgres, database postgres++4) Fill envs+- app/.env+ NEXT_PUBLIC_SUPABASE_URL=http://127.0.0.1:54321+ NEXT_PUBLIC_SUPABASE_ANON_KEY=<anon>+ SUPABASE_SERVICE_ROLE_KEY=<service_role>+ SUPABASE_PROJECT_ID=local+ SUPABASE_HOST=127.0.0.1+ SUPABASE_PORT=54322+ SUPABASE_DATABASE=postgres+ SUPABASE_USER=postgres+ SUPABASE_PASSWORD=postgres+- app/api/.env+ SUPABASE_URL=http://127.0.0.1:54321+ SUPABASE_KEY=<service_role>+ SUPABASE_HOST=127.0.0.1+ SUPABASE_PORT=54322+ SUPABASE_DATABASE=postgres+ SUPABASE_USER=postgres+ SUPABASE_PASSWORD=postgres+- app/dashboard/.env.local+ NEXT_PUBLIC_SUPABASE_URL=http://127.0.0.1:54321+ NEXT_PUBLIC_SUPABASE_ANON_KEY=<anon>+ SUPABASE_SERVICE_ROLE_KEY=<service_role>+ SUPABASE_PROJECT_ID=local++5) Run stack+- From app/: docker compose up -d+- API: http://localhost:8000/redoc+- Dashboard: http://localhost:3000++6) Notes+- Playground must be disabled:+ app/.env -> NEXT_PUBLIC_PLAYGROUND=false+ app/dashboard/.env.local -> NEXT_PUBLIC_PLAYGROUND="false"+- ClickHouse Cloud requires IP allowlist.