Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
Python agentagentopsagents-sdkaianthropicautogencost-estimationcrewaievalsevaluation-metricsgroqlangchainllmmistralollamaopenaiopenai-agents
AgentOps Handbook
Engineering and Product
"Great things in business are never done by one person. They're done by a team of people." — Steve Jobs
Goals of this handbook
- Stay as brief and short as possible
- Provide direction and procedure for product tasks
- Act as a living document, updated as we learn and hone in our practices
Contents
- Handling Incidents
- Contributing Code
- ticketing
- support
Acknowledgments
- Incident Review and Postmortem Best Practices from Pragmatic Engineer
- This handbook uses the Posthog handbook as a point of reference
- Much of that handbook was authored by @ellie (she's cool)