Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.
Python aiburrchatbot-frameworkdagsgenerative-aigraphshacktoberfestllmopsllmsmlopspersistent-data-structurestate-machinestate-managementvisibility
Adds notebook to typed state prototpye
elijahbenizzy6566eb48aa3e
diff --git a/examples/typed-state/__init__.py b/examples/typed-state/__init__.pynew file mode 100644index 00000000..e69de29bdiff --git a/examples/typed-state/application.py b/examples/typed-state/application.pyindex d737de97..88f1fad8 100644--- a/examples/typed-state/application.py+++ b/examples/typed-state/application.py@@ -1,4 +1,3 @@-import textwrapfrom typing import AsyncGenerator, Generator, Optional, Tuple, Unionimport instructor@@ -22,10 +21,6 @@ class Concept(BaseModel):definition: str = Field(description="A brief definition or explanation of the term.")timestamp: float = Field(description="Timestamp when the concept is explained.")- def display(self):- minutes, seconds = divmod(self.timestamp, 60)- return f"{int(minutes)}:{int(seconds)} - {self.term}: {self.definition}"-class SocialMediaPost(BaseModel):"""A social media post about a YouTube video generated its transcript"""@@ -51,28 +46,6 @@ class SocialMediaPost(BaseModel):)youtube_url: SkipJsonSchema[Union[str, None]] = None- def display(self) -> str:- formatted_takeways = " ".join([t for t in self.key_takeaways])- formatted_concepts = "CONCEPTS\n" + "\n".join([c.display() for c in self.concepts])- link = f"link: {self.youtube_url}\n\n" if self.youtube_url else ""-- return (- textwrap.dedent(- f"""\- TOPIC: {self.topic}-- {self.hook}-- {self.body}-- {formatted_takeways}-- """- )- + link- + formatted_concepts- )-class ApplicationState(BaseModel):# Make these have defaults as they are only set in actions@@ -93,8 +66,6 @@ def get_youtube_transcript(state: ApplicationState, youtube_url: str) -> Applicastate.transcript = " ".join([f"ts={entry['start']} - {entry['text']}" for entry in transcript])return state- # store the transcript in state-@action.pydantic(reads=["transcript"], writes=["post"])def generate_post(state: ApplicationState, llm_client) -> ApplicationState:diff --git a/examples/typed-state/notebook.ipynb b/examples/typed-state/notebook.ipynbindex cacaac34..bd14801e 100644--- a/examples/typed-state/notebook.ipynb+++ b/examples/typed-state/notebook.ipynb@@ -2,290 +2,241 @@"cells": [{"cell_type": "markdown",+ "id": "8419f04e-f945-491d-9526-6aebbabbad6a","metadata": {},"source": [- "# Build trustworthy LLM agents & applications for production with Instructor + Burr\n",+ "# Typed State\n","\n",- "The challenge with large language models (LLMs) is handling the 5% of the time they say crazy things. Being able to debug why an output is bad and having tools for fixing are critical requirements for making LLM features / agents trustworthy and available to users.\n",+ "In this example we're going to be using state-typing with instructor + Burr to generate social media posts from youtube videos.\n","\n",- "In this notebook, you'll learn how `instructor` can make LLM reliability produce structured outputs, and `burr` helps you introspect and debug your application."+ "First, let's define some pydantic models. Note you'll need the env var `OPENAI_API_KEY` set. "]},{- "cell_type": "markdown",- "metadata": {},+ "cell_type": "code",+ "execution_count": 22,+ "id": "d62b0737-683c-4748-8b55-c15402aa1b2f",+ "metadata": {+ "scrolled": true+ },+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Requirement already satisfied: burr[pydantic] in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (0.24.0)\n",+ "Requirement already satisfied: instructor in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (1.4.1)\n",+ "Requirement already satisfied: openai in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (1.44.0)\n",+ "Requirement already satisfied: rich in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (13.7.1)\n",+ "\u001b[33mWARNING: burr 0.24.0 does not provide the extra 'pydantic'\u001b[0m\u001b[33m\n",+ "\u001b[0mRequirement already satisfied: aiohttp<4.0.0,>=3.9.1 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (3.9.5)\n",+ "Requirement already satisfied: docstring-parser<0.17,>=0.16 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (0.16)\n",+ "Requirement already satisfied: jiter<0.6.0,>=0.5.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (0.5.0)\n",+ "Requirement already satisfied: pydantic<3.0.0,>=2.8.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (2.8.2)\n",+ "Requirement already satisfied: pydantic-core<3.0.0,>=2.18.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (2.20.1)\n",+ "Requirement already satisfied: tenacity<9.0.0,>=8.4.1 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (8.5.0)\n",+ "Requirement already satisfied: typer<1.0.0,>=0.9.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from instructor) (0.12.3)\n",+ "Requirement already satisfied: anyio<5,>=3.5.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from openai) (4.4.0)\n",+ "Requirement already satisfied: distro<2,>=1.7.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from openai) (1.9.0)\n",+ "Requirement already satisfied: httpx<1,>=0.23.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from openai) (0.27.0)\n",+ "Requirement already satisfied: sniffio in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from openai) (1.3.1)\n",+ "Requirement already satisfied: tqdm>4 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from openai) (4.66.4)\n",+ "Requirement already satisfied: typing-extensions<5,>=4.11 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from openai) (4.12.2)\n",+ "Requirement already satisfied: markdown-it-py>=2.2.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from rich) (3.0.0)\n",+ "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from rich) (2.18.0)\n",+ "Requirement already satisfied: aiosignal>=1.1.2 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from aiohttp<4.0.0,>=3.9.1->instructor) (1.3.1)\n",+ "Requirement already satisfied: attrs>=17.3.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from aiohttp<4.0.0,>=3.9.1->instructor) (23.2.0)\n",+ "Requirement already satisfied: frozenlist>=1.1.1 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from aiohttp<4.0.0,>=3.9.1->instructor) (1.4.1)\n",+ "Requirement already satisfied: multidict<7.0,>=4.5 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from aiohttp<4.0.0,>=3.9.1->instructor) (6.0.5)\n",+ "Requirement already satisfied: yarl<2.0,>=1.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from aiohttp<4.0.0,>=3.9.1->instructor) (1.9.4)\n",+ "Requirement already satisfied: idna>=2.8 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from anyio<5,>=3.5.0->openai) (3.7)\n",+ "Requirement already satisfied: certifi in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from httpx<1,>=0.23.0->openai) (2024.7.4)\n",+ "Requirement already satisfied: httpcore==1.* in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from httpx<1,>=0.23.0->openai) (1.0.5)\n",+ "Requirement already satisfied: h11<0.15,>=0.13 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai) (0.14.0)\n",+ "Requirement already satisfied: mdurl~=0.1 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from markdown-it-py>=2.2.0->rich) (0.1.2)\n",+ "Requirement already satisfied: annotated-types>=0.4.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from pydantic<3.0.0,>=2.8.0->instructor) (0.7.0)\n",+ "Requirement already satisfied: click>=8.0.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from typer<1.0.0,>=0.9.0->instructor) (8.1.7)\n",+ "Requirement already satisfied: shellingham>=1.3.0 in /Users/elijahbenizzy/.pyenv/versions/3.12.0/envs/burr-3-12/lib/python3.12/site-packages (from typer<1.0.0,>=0.9.0->instructor) (1.5.4)\n",+ "\n",+ "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",+ "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.12 -m pip install --upgrade pip\u001b[0m\n",+ "Note: you may need to restart the kernel to use updated packages.\n"+ ]+ }+ ],"source": [- "## Instructor 101\n",- "Instructor is a tool to help you prompt LLM and constraint its outputs. First, you specify the desired output using a **model** with typed fields and textual descriptions; you can think of it as a template that the LLM will fill. This greatly improves the reliability of the content and format of generated text.\n",- "\n",- "To introduce Instructor, we'll write code to generate a social media post from the transcript of a YouTube video. \n",- "\n",- "> This post on the Instructor blog is also a great introduction: [Analyzing Youtube Transcripts with Instructor](https://python.useinstructor.com/blog/2024/07/11/youtube-transcripts/)."+ "%pip install 'burr[pydantic]' instructor openai rich"]},{"cell_type": "markdown",+ "id": "66fe6bbc-3c75-4ff6-9974-4d82109dc47a","metadata": {},"source": [- "### 1. Define the `response_model`\n",- "\n",- "Instructor uses [Pydantic](https://docs.pydantic.dev/latest/) to create the response model. A model needs to inherit the `BaseModel` class and we use the `Field()` object to give a textual description.\n",- "\n",- "- `Field()` objects allow to specify constraints to the generated output. For instance, we want \"1 to 3 concepts\" and \"1 to 4 key takeaways\" generated per `SocialMediaPost`\n",- "- Notice that you can nest models. Indeed, `SocialMediaPost.concepts` is a list of `Concept` models.\n",- "- We use `SkipJsonSchema` on the `youtube_url` field to specify that this shouldn't be generated by the LLM. Instead, we'll manually pass it.\n",- "\n",- "Tip: We're adding a `.display()` method to format the text to be more easily human-readable. "+ "# Imports/setup"]},{"cell_type": "code",- "execution_count": 1,+ "execution_count": 26,+ "id": "c3da83a6-0047-4599-aa0c-c4d7a0cc7e78","metadata": {},"outputs": [],"source": [- "import textwrap\n",- "from typing import Union\n",+ "from typing import AsyncGenerator, Generator, Optional, Tuple, Union\n","\n",+ "import instructor\n",+ "import openai\n","from pydantic import BaseModel, Field\n","from pydantic.json_schema import SkipJsonSchema\n",+ "from rich.console import Console\n",+ "from youtube_transcript_api import YouTubeTranscriptApi\n","\n",- "class Concept(BaseModel):\n",- " term: str = Field(description=\"A key term or concept mentioned.\")\n",- " definition: str = Field(description=\"A brief definition or explanation of the term.\")\n",- " timestamp: float = Field(description=\"Timestamp when the concept is explained.\")\n",- "\n",- " def display(self):\n",- " minutes, seconds = divmod(self.timestamp, 60)\n",- " return f\"{int(minutes)}:{int(seconds)} - {self.term}: {self.definition}\"\n",- "\n",- "\n",- "class SocialMediaPost(BaseModel):\n",- " \"\"\"A social media post about a YouTube video generated its transcript\"\"\"\n",- "\n",- " topic: str = Field(description=\"Main topic discussed.\")\n",- " hook: str = Field(description=\"Statement to grab the attention of the reader and announce the topic.\")\n",- " body: str = Field(description=\"The body of the social media post. It should be informative and make the reader curious about viewing the video.\")\n",- " concepts: list[Concept] = Field(\n",- " description=\"Important concepts about Hamilton or Burr mentioned in this post.\",\n",- " min_items=1,\n",- " max_items=3,\n",- " )\n",- " key_takeaways: list[str] = Field(\n",- " description=\"A list of informative key takeways for the reader.\",\n",- " min_items=1,\n",- " max_items=4,\n",- " )\n",- " youtube_url: SkipJsonSchema[Union[str, None]] = None\n",- "\n",- " def display(self) -> str:\n",- " formatted_takeways = \" \".join([t for t in self.key_takeaways])\n",- " formatted_concepts = \"CONCEPTS\\n\" + \"\\n\".join([c.display() for c in self.concepts])\n",- " link = f\"link: {self.youtube_url}\\n\\n\" if self.youtube_url else \"\"\n",- "\n",- " return textwrap.dedent(\n",- " f\"\"\"\\\n",- " TOPIC: {self.topic}\n",- "\n",- " {self.hook}\n",- "\n",- " {self.body}\n",- "\n",- " {formatted_takeways}\n",- "\n",- " \"\"\"\n",- " ) + link + formatted_concepts"+ "from burr.core import Application, ApplicationBuilder, action\n",+ "from burr.core.action import (\n",+ " AsyncStreamingResultContainer,\n",+ " StreamingResultContainer,\n",+ " streaming_action,\n",+ ")\n",+ "from burr.integrations.pydantic import PydanticTypingSystem\n",+ "import json\n",+ "import time\n",+ "from rich import print_json\n",+ "from IPython.display import clear_output"]},{"cell_type": "markdown",+ "id": "1c5d6d06-3a82-44df-b4fe-82c95b3d3d05","metadata": {},"source": [- "### 2. Write the application logic\n",+ "# Constructs\n","\n",- "Instructor is not opiniated about how you write your application; it's only in contact with your LLM client. Here, we write a script in a few lines of code to retrieve a YouTube transcript."+ "Let's define some pydantic models to use -- these will help shape our application"]},{"cell_type": "code",- "execution_count": 2,+ "execution_count": 11,+ "id": "90258678-b33e-4e84-9a22-c7a6d29aca5d","metadata": {},"outputs": [],"source": [- "from youtube_transcript_api import YouTubeTranscriptApi\n",+ "class Concept(BaseModel):\n",+ " term: str = Field(description=\"A key term or concept mentioned.\")\n",+ " definition: str = Field(description=\"A brief definition or explanation of the term.\")\n",+ " timestamp: float = Field(description=\"Timestamp when the concept is explained.\")\n","\n",- "# get the video id from a YouTube url\n",- "youtube_url = \"https://www.youtube.com/watch?v=hqutVJyd3TI\" \n",- "_, _, video_id = youtube_url.partition(\"?v=\")\n",+ "class SocialMediaPost(BaseModel):\n",+ " \"\"\"A social media post about a YouTube video generated its transcript\"\"\"\n","\n",- "# get the available YouTube transcript for the video\n",- "transcript = YouTubeTranscriptApi.get_transcript(video_id)\n",- "# join the transcript into a single block of text\n",- "full_transcript = \" \".join([f\"ts={entry['start']} - {entry['text']}\" for entry in transcript])"+ " topic: str = Field(description=\"Main topic discussed.\")\n",+ " hook: str = Field(\n",+ " description=\"Statement to grab the attention of the reader and announce the topic.\"\n",+ " )\n",+ " body: str = Field(\n",+ " description=\"The body of the social media post. It should be informative and make the reader curious about viewing the video.\"\n",+ " )\n",+ " concepts: list[Concept] = Field(\n",+ " description=\"Important concepts about Hamilton or Burr mentioned in this post -- please have at least 1\",\n",+ " min_items=0,\n",+ " max_items=3,\n",+ " validate_default=False,\n",+ " )\n",+ " key_takeaways: list[str] = Field(\n",+ " description=\"A list of informative key takeways for the reader -- please have at least 1\",\n",+ " min_items=0,\n",+ " max_items=4,\n",+ " validate_default=False,\n",+ " )\n",+ " youtube_url: SkipJsonSchema[Union[str, None]] = None"]},{"cell_type": "markdown",+ "id": "03534c74-aa1f-4a1d-91c7-7c6328228c8d","metadata": {},"source": [- "### Wrap the LLM client\n",- "To use Instructor, we need to wrap the OpenAI client, creating a special client. \n",+ "# State Type\n","\n",- "> NOTE: If you have the environment variable `OPENAI_API_KEY` set, the client will be automatically created. Otherwise, you'll need to manually pass the key to `OpenAI(api_key=...)`."+ "Using those, we'll define a core pydantic model that sets up the central schema for our application. Note these are optional, they won't be set when our application starts!"]},{"cell_type": "code",- "execution_count": 5,+ "execution_count": 12,+ "id": "5158c112-f054-4881-89dd-12ca7f1e0e79","metadata": {},"outputs": [],"source": [- "import instructor\n",- "from openai import OpenAI\n",- "\n",- "llm_client = instructor.from_openai(OpenAI())"+ "class ApplicationState(BaseModel):\n",+ " # Make these have defaults as they are only set in actions\n",+ " transcript: Optional[str] = Field(\n",+ " description=\"The full transcript of the YouTube video.\", default=None\n",+ " )\n",+ " post: Optional[SocialMediaPost] = Field(\n",+ " description=\"The generated social media post.\", default=None\n",+ " )"]},{"cell_type": "markdown",+ "id": "f8e0d745-6561-4331-9a5d-86948fee706d","metadata": {},"source": [- "#### Use the LLM client with the `response_model`\n",+ "# Write an action to transcribe a youtube URL\n","\n",- "1. Use the LLM client with `.create` to call the LLM API\n",- "2. Pass `SocialMediaPost` as the response model, enabling structured outputs.\n",- "3. The `messages` include the `system` message with the task instruction for the LLM and\n",- "the `user` message with the input content. "+ "Note we take in a youtube URL + the state in the format we want, and write to `transcript`. We actually read nothing, as the transcript is an input.\n",+ "Different than normal Burr, we actually mutate the model we send in (this allows us to leverage pydantic validation)."]},{"cell_type": "code",- "execution_count": 6,+ "execution_count": 16,+ "id": "df35e278-3578-4611-a9de-c3948911ff70","metadata": {},"outputs": [],"source": [- "response = llm_client.chat.completions.create(\n",- " model=\"gpt-4o-mini\",\n",- " response_model=SocialMediaPost,\n",- " messages=[\n",- " {\n",- " \"role\": \"system\",\n",- " \"content\": \"Analyze the given YouTube transcript and generate a compelling social media post.\",\n",- " },\n",- " {\"role\": \"user\", \"content\": full_transcript},\n",- " ],\n",- ")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "`response` will have the type of the provided `response_model`, `SocialMediaPost` in this case. You can use `Model.model_dump()` to get a Python dictionary."- ]- },- {- "cell_type": "code",- "execution_count": 7,- "metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "<class '__main__.SocialMediaPost'>\n",- "{'topic': 'Understanding B: A Powerful Tool for Agent Applications', 'hook': '🚀 Debug with ease using B!', 'body': 'Ever faced frustrating issues while building agent applications? Dive into our latest video where we unveil the capabilities of B, a framework designed to enhance observability and debugging. Learn how to fix bugs without restarting your entire agent and explore B’s unique graph-building capabilities for a seamless coding experience. Perfect for developers looking to streamline their coding processes!', 'concepts': [{'term': 'Agent Applications', 'definition': 'Applications that utilize AI agents to perform tasks or make decisions based on input and graphs of actions.', 'timestamp': 108.0}, {'term': 'Graph Building', 'definition': 'Creating a flowchart-like structure of actions and states for better process management in applications.', 'timestamp': 179.0}, {'term': 'Local Tracker', 'definition': 'A feature within B that allows interaction with the application state for tracking and debugging purposes.', 'timestamp': 260.0}], 'key_takeaways': ['B enables easy bug fixing without restarting your agent.', 'Graph-based modeling simplifies the coding process.', 'Integrate local tracking for comprehensive observability in applications.'], 'youtube_url': None}\n"- ]- }- ],- "source": [- "print(type(response))\n",- "print(response.model_dump())"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "We can also use the `.display()` method we've defined!"- ]- },- {- "cell_type": "code",- "execution_count": 8,- "metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "TOPIC: Understanding B: A Powerful Tool for Agent Applications\n",- "\n",- "🚀 Debug with ease using B!\n",- "\n",- "Ever faced frustrating issues while building agent applications? Dive into our latest video where we unveil the capabilities of B, a framework designed to enhance observability and debugging. Learn how to fix bugs without restarting your entire agent and explore B’s unique graph-building capabilities for a seamless coding experience. Perfect for developers looking to streamline their coding processes!\n",- "\n",- "B enables easy bug fixing without restarting your agent. Graph-based modeling simplifies the coding process. Integrate local tracking for comprehensive observability in applications.\n",- "\n",- "CONCEPTS\n",- "1:48 - Agent Applications: Applications that utilize AI agents to perform tasks or make decisions based on input and graphs of actions.\n",- "2:59 - Graph Building: Creating a flowchart-like structure of actions and states for better process management in applications.\n",- "4:20 - Local Tracker: A feature within B that allows interaction with the application state for tracking and debugging purposes.\n"- ]- }- ],- "source": [- "print(response.display())"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Burr 101\n",- "Burr is a tool to build LLM applications, solving many challenges to get to production (monitoring, persistence, streaming, and more). With the concepts of \"state\" and \"action\", you can define complex apps that are easy-to-understand and debug.\n",+ "@action.pydantic(reads=[], writes=[\"transcript\"])\n",+ "def get_youtube_transcript(state: ApplicationState, youtube_url: str) -> ApplicationState:\n",+ " \"\"\"Get the official YouTube transcript for a video given its URL\"\"\"\n",+ " _, _, video_id = youtube_url.partition(\"?v=\")\n","\n",- "To show this, we'll rewrite the previous application logic with Burr."+ " transcript = YouTubeTranscriptApi.get_transcript(video_id, languages=[\"en\"])\n",+ " state.transcript = \" \".join([f\"ts={entry['start']} - {entry['text']}\" for entry in transcript])\n",+ " return state"]},{"cell_type": "markdown",+ "id": "f5dbcaf5-5471-44b0-82e3-c4b32f01a115","metadata": {},"source": [- "### 1. Define `actions`\n",+ "# Write an action to stream back pydantic models\n","\n",- "First, you need to define the different actions your agent can take. This is done by writing Python functions with the `@action` decorator. The decorator must specify the information that can be read from state. Also, the function needs to take a `State` object as first argument and return a `State` object."+ "We specify the state input type, state output type, and the stream type, streaming it all back using [instructors streaming capability](https://python.useinstructor.com/concepts/partial/)"]},{"cell_type": "code",- "execution_count": 9,+ "execution_count": 18,+ "id": "4737d8b1-e1c1-472c-bc01-a47d328797d8","metadata": {},"outputs": [],"source": [- "from burr.core import State, action\n",- "\n",- "\n",- "@action(reads=[], writes=[\"transcript\"])\n",- "def get_youtube_transcript(state: State, youtube_url: str) -> State:\n",- " \"\"\"Get the official YouTube transcript for a video given it's URL\"\"\"\n",- " _, _, video_id = youtube_url.partition(\"?v=\")\n",- " \n",- " transcript = YouTubeTranscriptApi.get_transcript(video_id, languages=[\"en\"])\n",- " full_transcript = \" \".join([f\"ts={entry['start']} - {entry['text']}\" for entry in transcript])\n",- "\n",- " # store the transcript in state\n",- " return state.update(transcript=full_transcript, youtube_url=youtube_url)\n",- "\n",- "\n",- "@action(reads=[\"transcript\"], writes=[\"post\"])\n",- "def generate_post(state: State, llm_client) -> State:\n",- " \"\"\"Use the Instructor LLM client to generate `SocialMediaPost` from the YouTube transcript.\"\"\"\n",- "\n",- " # read the transcript from state\n",- " transcript = state[\"transcript\"]\n",+ "@streaming_action.pydantic(\n",+ " reads=[\"transcript\"],\n",+ " writes=[\"post\"],\n",+ " state_input_type=ApplicationState,\n",+ " state_output_type=ApplicationState,\n",+ " stream_type=SocialMediaPost,\n",+ ")\n",+ "def generate_post(\n",+ " state: ApplicationState, llm_client\n",+ ") -> Generator[Tuple[SocialMediaPost, Optional[ApplicationState]], None, None]:\n","\n",- " response = llm_client.chat.completions.create(\n",+ " transcript = state.transcript\n",+ " response = llm_client.chat.completions.create_partial(\n"," model=\"gpt-4o-mini\",\n"," response_model=SocialMediaPost,\n"," messages=[\n",@@ -295,29 +246,30 @@" },\n"," {\"role\": \"user\", \"content\": transcript},\n"," ],\n",+ " stream=True,\n"," )\n",+ " final_post: SocialMediaPost = None # type: ignore\n",+ " for post in response:\n",+ " final_post = post\n",+ " yield post, None\n","\n",- " # add the youtube_url found in state to the SocialMediaPost\n",- " response.youtube_url = state[\"youtube_url\"]\n",- "\n",- " # store the chapters in state\n",- " return state.update(post=response)"+ " yield final_post, state"]},{"cell_type": "markdown",+ "id": "968daa7a-c83f-4890-b775-513dbc347068","metadata": {},"source": [- "### 2. Assemble the `Application`\n",- "\n",- "To create a Burr agent, we need to assemble the `actions` into an `Application`. This requires specifying the valid `transitions` between actions using tuples of action names `(from, to)` and defining an `entrypoint` from where to begin execution. Then, we can visualize the graph of possible states and actions.\n",+ "# Wire together in an application\n","\n",- "Notice that we create the Instructor LLM client *outside* the application and pass it to the `generate_chapters` action via the `.bind()` method. This method follows the same logic as the standard library `functools.partial()`."+ "We specify the application to have type `ApplicationState` as the state, and pass it an initial value"]},{"cell_type": "code",- "execution_count": 10,+ "execution_count": 24,+ "id": "11bdf44a-5ff2-4cd3-87b9-175eb68de8e4","metadata": {},"outputs": [{@@ -326,151 +278,64 @@"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n","<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n"," \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",- "<!-- Generated by graphviz version 2.43.0 (0)\n",+ "<!-- Generated by graphviz version 12.0.0 (20240704.0754)\n"," -->\n",- "<!-- Title: %3 Pages: 1 -->\n",- "<svg width=\"200pt\" height=\"174pt\"\n",- " viewBox=\"0.00 0.00 200.38 174.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",- "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 170)\">\n",- "<title>%3</title>\n",- "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-170 196.38,-170 196.38,4 -4,4\"/>\n",+ "<!-- Pages: 1 -->\n",+ "<svg width=\"167pt\" height=\"176pt\"\n",+ " viewBox=\"0.00 0.00 166.85 175.80\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",+ "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 171.8)\">\n",+ "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-171.8 162.85,-171.8 162.85,4 -4,4\"/>\n","<!-- get_youtube_transcript -->\n","<g id=\"node1\" class=\"node\">\n","<title>get_youtube_transcript</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M175.19,-101C175.19,-101 17.19,-101 17.19,-101 11.19,-101 5.19,-95 5.19,-89 5.19,-89 5.19,-77 5.19,-77 5.19,-71 11.19,-65 17.19,-65 17.19,-65 175.19,-65 175.19,-65 181.19,-65 187.19,-71 187.19,-77 187.19,-77 187.19,-89 187.19,-89 187.19,-95 181.19,-101 175.19,-101\"/>\n",- "<text text-anchor=\"middle\" x=\"96.19\" y=\"-79.3\" font-family=\"Times,serif\" font-size=\"14.00\">get_youtube_transcript</text>\n",+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M146.85,-102.2C146.85,-102.2 12,-102.2 12,-102.2 6,-102.2 0,-96.2 0,-90.2 0,-90.2 0,-77.6 0,-77.6 0,-71.6 6,-65.6 12,-65.6 12,-65.6 146.85,-65.6 146.85,-65.6 152.85,-65.6 158.85,-71.6 158.85,-77.6 158.85,-77.6 158.85,-90.2 158.85,-90.2 158.85,-96.2 152.85,-102.2 146.85,-102.2\"/>\n",+ "<text text-anchor=\"middle\" x=\"79.42\" y=\"-78.1\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">get_youtube_transcript</text>\n","</g>\n","<!-- generate_post -->\n","<g id=\"node3\" class=\"node\">\n","<title>generate_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M143.19,-36C143.19,-36 49.19,-36 49.19,-36 43.19,-36 37.19,-30 37.19,-24 37.19,-24 37.19,-12 37.19,-12 37.19,-6 43.19,0 49.19,0 49.19,0 143.19,0 143.19,0 149.19,0 155.19,-6 155.19,-12 155.19,-12 155.19,-24 155.19,-24 155.19,-30 149.19,-36 143.19,-36\"/>\n",- "<text text-anchor=\"middle\" x=\"96.19\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">generate_post</text>\n",+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M121.35,-36.6C121.35,-36.6 37.5,-36.6 37.5,-36.6 31.5,-36.6 25.5,-30.6 25.5,-24.6 25.5,-24.6 25.5,-12 25.5,-12 25.5,-6 31.5,0 37.5,0 37.5,0 121.35,0 121.35,0 127.35,0 133.35,-6 133.35,-12 133.35,-12 133.35,-24.6 133.35,-24.6 133.35,-30.6 127.35,-36.6 121.35,-36.6\"/>\n",+ "<text text-anchor=\"middle\" x=\"79.42\" y=\"-12.5\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">generate_post</text>\n","</g>\n","<!-- get_youtube_transcript->generate_post -->\n","<g id=\"edge2\" class=\"edge\">\n","<title>get_youtube_transcript->generate_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M96.19,-64.78C96.19,-59.09 96.19,-52.61 96.19,-46.34\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"99.69,-46.19 96.19,-36.19 92.69,-46.19 99.69,-46.19\"/>\n",+ "<path fill=\"none\" stroke=\"black\" d=\"M73.3,-65.18C72.8,-59.95 72.6,-54.07 72.7,-48.32\"/>\n",+ "<polygon fill=\"black\" stroke=\"black\" points=\"76.19,-48.61 73.23,-38.43 69.2,-48.23 76.19,-48.61\"/>\n","</g>\n","<!-- input__youtube_url -->\n","<g id=\"node2\" class=\"node\">\n","<title>input__youtube_url</title>\n",- "<ellipse fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" cx=\"96.19\" cy=\"-148\" rx=\"96.38\" ry=\"18\"/>\n",- "<text text-anchor=\"middle\" x=\"96.19\" y=\"-144.3\" font-family=\"Times,serif\" font-size=\"14.00\">input: youtube_url</text>\n",+ "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"143.85,-167.8 15,-167.8 15,-131.2 143.85,-131.2 143.85,-167.8\"/>\n",+ "<text text-anchor=\"middle\" x=\"79.42\" y=\"-143.7\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">input: youtube_url</text>\n","</g>\n","<!-- input__youtube_url->get_youtube_transcript -->\n","<g id=\"edge1\" class=\"edge\">\n","<title>input__youtube_url->get_youtube_transcript</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M96.19,-129.78C96.19,-124.09 96.19,-117.61 96.19,-111.34\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"99.69,-111.19 96.19,-101.19 92.69,-111.19 99.69,-111.19\"/>\n",+ "<path fill=\"none\" stroke=\"black\" d=\"M79.42,-130.78C79.42,-125.55 79.42,-119.67 79.42,-113.92\"/>\n",+ "<polygon fill=\"black\" stroke=\"black\" points=\"82.93,-114.03 79.43,-104.03 75.93,-114.03 82.93,-114.03\"/>\n",+ "</g>\n",+ "<!-- generate_post->get_youtube_transcript -->\n",+ "<g id=\"edge3\" class=\"edge\">\n",+ "<title>generate_post->get_youtube_transcript</title>\n",+ "<path fill=\"none\" stroke=\"black\" d=\"M85.54,-36.92C86.04,-42.15 86.25,-48.02 86.15,-53.78\"/>\n",+ "<polygon fill=\"black\" stroke=\"black\" points=\"82.66,-53.5 85.63,-63.67 89.65,-53.87 82.66,-53.5\"/>\n","</g>\n","</g>\n","</svg>\n"],"text/plain": [- "<graphviz.graphs.Digraph at 0x7f94ead0d270>"+ "<burr.core.application.Application at 0x12fff20c0>"]},- "execution_count": 10,+ "execution_count": 24,"metadata": {},"output_type": "execute_result"}],"source": [- "from burr.core import ApplicationBuilder\n",- "\n",- "application = (\n",- " ApplicationBuilder()\n",- " .with_actions(\n",- " get_youtube_transcript,\n",- " generate_post.bind(llm_client=llm_client),\n",- " )\n",- " .with_transitions(\n",- " (\"get_youtube_transcript\", \"generate_post\"),\n",- " )\n",- " .with_entrypoint(\"get_youtube_transcript\")\n",- " .build()\n",- ")\n",- "application.visualize()"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "### 3. Launch the application\n",- "\n",- "Using `application.run()` will make our application iterate through actions and state until it hits a `halt` condition. Here, we will simply halt after completing the `generate_post` action. This will return a tuple of (the last action take, the result of the last action, the state of the app). We also need to pass a `youtube_url` since it's a required input to the `get_youtube_transcript` action."- ]- },- {- "cell_type": "code",- "execution_count": 11,- "metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "TOPIC: Debugging with Burr and Graph States\n",- "\n",- "Ever struggled with debugging your AI agents?\n",- "\n",- "Dive into the world of Burr with our latest video! Learn how to leverage state tracking and condition-based flowcharts to handle failures and streamline debugging like never before. Discover how you can monitor your agents, and even resume operations from a specific point in their execution—without starting over! 🚀 Whether you’re a developer building complex applications or simply curious about AI functionality, this video will enhance your understanding of agent workflows. Don't miss out!\n",- "\n",- "Learn how to handle failures in AI agent applications effectively. Understand the relationship between actions and state in your applications. Discover how to resume operations from a specific state without restarting the entire application.\n",- "\n",- "link: https://www.youtube.com/watch?v=hqutVJyd3TI\n",- "\n",- "CONCEPTS\n",- "10:57 - State Tracking: State tracking allows you to monitor the state of your application at different points in time, making debugging easier.\n",- "1:50 - Flowchart: In this context, a flowchart represents the actions and their dependencies in your agent application as a directed graph.\n",- "3:36 - Modality: The condition that determines which action to take next in the flow of your agent’s tasks.\n"- ]- }- ],- "source": [- "last_action, result, state = application.run(\n",- " halt_after=[\"generate_post\"],\n",- " inputs={\"youtube_url\": \"https://www.youtube.com/watch?v=hqutVJyd3TI\"},\n",- ")\n",- "\n",- "# print `post` stored in state\n",- "print(state[\"post\"].display())"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "## Why create a Burr application?\n",- "\n",- "In a few lines of code, you can query an LLM API and can create powerful productivity utilities. However, user-facing features deserve much more scrutiny, which requires tooling and solving complex engineering problems.\n",- "\n",- "Building our app with Burr provides several benefits that we'll detail next:\n",- "- **Observability**: monitor in real-time and log the execution of your `Application` and view it in Burr's web user interface.\n",- "- **Persistence**: At any point, you can save the application `State`. This allows to create user sessions (e.g., the conversation history menu in ChatGPT), which helps developers investigate bugs and test potential solutions.\n",- "- **Portability**: your `Application` can run in a notebook, as a script, as a web service, or anywhere Python runs. We'll show how to use Burr with [FastAPI](https://fastapi.tiangolo.com/)."- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "### 1. Observability\n",- "\n",- "Add the clause `.with_tracker(project=...)` to the `ApplicationBuilder()` to track execution. \n"- ]- },- {- "cell_type": "code",- "execution_count": 11,- "metadata": {},- "outputs": [],- "source": [- "from burr.core import ApplicationBuilder\n",- "\n",- "application = (\n",+ "llm_client = instructor.from_openai(openai.OpenAI())\n",+ "app = (\n"," ApplicationBuilder()\n"," .with_actions(\n"," get_youtube_transcript,\n",@@ -478,337 +343,114 @@" )\n"," .with_transitions(\n"," (\"get_youtube_transcript\", \"generate_post\"),\n",+ " (\"generate_post\", \"get_youtube_transcript\"),\n"," )\n"," .with_entrypoint(\"get_youtube_transcript\")\n",+ " .with_typing(PydanticTypingSystem(ApplicationState))\n",+ " .with_state(ApplicationState())\n"," .with_tracker(project=\"youtube-post\")\n"," .build()\n",- ")"- ]- },- {- "cell_type": "code",- "execution_count": null,- "metadata": {},- "outputs": [],- "source": [- "last_action, result, state = application.run(\n",- " halt_after=[\"generate_post\"],\n",- " inputs={\"youtube_url\": \"https://www.youtube.com/watch?v=hqutVJyd3TI\"},\n",- ")"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "Then, you can launch the web UI via the CLI command `burr`"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "### 2. Persistence\n",- "\n",- "To showcase this feature, we'll add a `rewrite()` action. It sends to the LLM the social media post and a user input to tweak its content. "- ]- },- {- "cell_type": "code",- "execution_count": 13,- "metadata": {},- "outputs": [],- "source": [- "@action(reads=[\"post\"], writes=[\"post\"])\n",- "def rewrite_post(state: State, llm_client, user_prompt: str):\n",- " post = state[\"post\"]\n",- "\n",- " response = llm_client.chat.completions.create(\n",- " model=\"gpt-4o-mini\",\n",- " response_model=SocialMediaPost,\n",- " messages=[\n",- " {\n",- " \"role\": \"system\",\n",- " \"content\": f\"Take the previously generated social media post and modify it according to the following instructions: {user_prompt}\",\n",- " },\n",- " {\"role\": \"user\", \"content\": post.model_dump_json()},\n",- " ],\n",- " )\n",- "\n",- " # pass the youtube_url from the previous post version\n",- " response.youtube_url = post.youtube_url\n",- "\n",- " return state.update(post=response)"+ ")\n",+ "# in case we want to access the state\n",+ "assert isinstance(app.state.data, ApplicationState)\n",+ "app"]},{"cell_type": "markdown",+ "id": "29eb49bc-f1c0-4172-831d-bd748ac90548","metadata": {},"source": [- "By adding the transition `(\"rewrite_post\", \"rewrite_post\")`, we are introducing a graph cycle. Observability and persistence becomes particularly valuable to ensure that the LLM doesn't spiral into non-sense. If that's the case, it could be due to the prompts / instructions in the application code, but also user inputs.\n",+ "# Run it!\n","\n",- "We also add a `.with_persister()` clause to store our results."+ "Now we can run it!"]},{"cell_type": "code",- "execution_count": 22,+ "execution_count": 32,+ "id": "ba1486aa-f79f-4e02-959c-ffc0ba58b6ac","metadata": {},"outputs": [{"data": {- "image/svg+xml": [- "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",- "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",- " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",- "<!-- Generated by graphviz version 2.43.0 (0)\n",- " -->\n",- "<!-- Title: %3 Pages: 1 -->\n",- "<svg width=\"381pt\" height=\"239pt\"\n",- " viewBox=\"0.00 0.00 381.28 239.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",- "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 235)\">\n",- "<title>%3</title>\n",- "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-235 377.28,-235 377.28,4 -4,4\"/>\n",- "<!-- get_youtube_transcript -->\n",- "<g id=\"node1\" class=\"node\">\n",- "<title>get_youtube_transcript</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M356.09,-166C356.09,-166 198.09,-166 198.09,-166 192.09,-166 186.09,-160 186.09,-154 186.09,-154 186.09,-142 186.09,-142 186.09,-136 192.09,-130 198.09,-130 198.09,-130 356.09,-130 356.09,-130 362.09,-130 368.09,-136 368.09,-142 368.09,-142 368.09,-154 368.09,-154 368.09,-160 362.09,-166 356.09,-166\"/>\n",- "<text text-anchor=\"middle\" x=\"277.09\" y=\"-144.3\" font-family=\"Times,serif\" font-size=\"14.00\">get_youtube_transcript</text>\n",- "</g>\n",- "<!-- generate_post -->\n",- "<g id=\"node3\" class=\"node\">\n",- "<title>generate_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M324.09,-101C324.09,-101 230.09,-101 230.09,-101 224.09,-101 218.09,-95 218.09,-89 218.09,-89 218.09,-77 218.09,-77 218.09,-71 224.09,-65 230.09,-65 230.09,-65 324.09,-65 324.09,-65 330.09,-65 336.09,-71 336.09,-77 336.09,-77 336.09,-89 336.09,-89 336.09,-95 330.09,-101 324.09,-101\"/>\n",- "<text text-anchor=\"middle\" x=\"277.09\" y=\"-79.3\" font-family=\"Times,serif\" font-size=\"14.00\">generate_post</text>\n",- "</g>\n",- "<!-- get_youtube_transcript->generate_post -->\n",- "<g id=\"edge3\" class=\"edge\">\n",- "<title>get_youtube_transcript->generate_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M277.09,-129.78C277.09,-124.09 277.09,-117.61 277.09,-111.34\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"280.59,-111.19 277.09,-101.19 273.59,-111.19 280.59,-111.19\"/>\n",- "</g>\n",- "<!-- input__youtube_url -->\n",- "<g id=\"node2\" class=\"node\">\n",- "<title>input__youtube_url</title>\n",- "<ellipse fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" cx=\"277.09\" cy=\"-213\" rx=\"96.38\" ry=\"18\"/>\n",- "<text text-anchor=\"middle\" x=\"277.09\" y=\"-209.3\" font-family=\"Times,serif\" font-size=\"14.00\">input: youtube_url</text>\n",- "</g>\n",- "<!-- input__youtube_url->get_youtube_transcript -->\n",- "<g id=\"edge1\" class=\"edge\">\n",- "<title>input__youtube_url->get_youtube_transcript</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M277.09,-194.78C277.09,-189.09 277.09,-182.61 277.09,-176.34\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"280.59,-176.19 277.09,-166.19 273.59,-176.19 280.59,-176.19\"/>\n",- "</g>\n",- "<!-- rewrite_post -->\n",- "<g id=\"node4\" class=\"node\">\n",- "<title>rewrite_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M229.59,-36C229.59,-36 146.59,-36 146.59,-36 140.59,-36 134.59,-30 134.59,-24 134.59,-24 134.59,-12 134.59,-12 134.59,-6 140.59,0 146.59,0 146.59,0 229.59,0 229.59,0 235.59,0 241.59,-6 241.59,-12 241.59,-12 241.59,-24 241.59,-24 241.59,-30 235.59,-36 229.59,-36\"/>\n",- "<text text-anchor=\"middle\" x=\"188.09\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">rewrite_post</text>\n",- "</g>\n",- "<!-- generate_post->rewrite_post -->\n",- "<g id=\"edge4\" class=\"edge\">\n",- "<title>generate_post->rewrite_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M252.77,-64.78C242.86,-57.77 231.26,-49.56 220.64,-42.04\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"222.57,-39.12 212.38,-36.19 218.52,-44.83 222.57,-39.12\"/>\n",- "</g>\n",- "<!-- rewrite_post->rewrite_post -->\n",- "<g id=\"edge5\" class=\"edge\">\n",- "<title>rewrite_post->rewrite_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M241.72,-29.47C252.17,-28.13 259.59,-24.31 259.59,-18 259.59,-13.96 256.55,-10.93 251.61,-8.93\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"252.27,-5.49 241.72,-6.53 250.62,-12.29 252.27,-5.49\"/>\n",- "</g>\n",- "<!-- input__user_prompt -->\n",- "<g id=\"node5\" class=\"node\">\n",- "<title>input__user_prompt</title>\n",- "<ellipse fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" cx=\"100.09\" cy=\"-83\" rx=\"100.18\" ry=\"18\"/>\n",- "<text text-anchor=\"middle\" x=\"100.09\" y=\"-79.3\" font-family=\"Times,serif\" font-size=\"14.00\">input: user_prompt</text>\n",- "</g>\n",- "<!-- input__user_prompt->rewrite_post -->\n",- "<g id=\"edge2\" class=\"edge\">\n",- "<title>input__user_prompt->rewrite_post</title>\n",- "<path fill=\"none\" stroke=\"black\" d=\"M123.21,-65.45C133.18,-58.31 145,-49.85 155.81,-42.11\"/>\n",- "<polygon fill=\"black\" stroke=\"black\" points=\"158.11,-44.77 164.2,-36.1 154.04,-39.08 158.11,-44.77\"/>\n",- "</g>\n",- "</g>\n",- "</svg>\n"+ "text/html": [+ "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">{</span>\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"topic\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"Burr Framework Overview\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"hook\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"Ever faced challenges while debugging AI applications? Here's a solution!\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"body\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"Dive into the world of agent applications with Burr! In this quick overview, we explore how Burr helps you debug failing AI calls and track state effectively. Learn to build a graph that connects actions and states, allowing you to resume your application exactly where you left off. Whether it’s fixing an error mid-run or replaying past actions, Burr's observability features enhance your development workflow. Ready to optimize your debugging process? Check out the full video to unravel the potential of Burr!\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"concepts\"</span>: <span style=\"font-weight: bold\">[</span>\n",+ " <span style=\"font-weight: bold\">{</span>\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"term\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"Agent Application\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"definition\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"A system that models states and actions to create decision-making processes.\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"timestamp\"</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">105.479</span>\n",+ " <span style=\"font-weight: bold\">}</span>,\n",+ " <span style=\"font-weight: bold\">{</span>\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"term\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"State Object\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"definition\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"An object that holds the state information for actions to read and write during execution.\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"timestamp\"</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">145.56</span>\n",+ " <span style=\"font-weight: bold\">}</span>,\n",+ " <span style=\"font-weight: bold\">{</span>\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"term\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"Graph Representation\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"definition\"</span>: <span style=\"color: #008000; text-decoration-color: #008000\">\"A structural representation of actions and their interconnections in an agent system, depicted as nodes and edges.\"</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"timestamp\"</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">179.28</span>\n",+ " <span style=\"font-weight: bold\">}</span>\n",+ " <span style=\"font-weight: bold\">]</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"key_takeaways\"</span>: <span style=\"font-weight: bold\">[</span>\n",+ " <span style=\"color: #008000; text-decoration-color: #008000\">\"Burr allows near-instantaneous debugging without restarting from scratch.\"</span>,\n",+ " <span style=\"color: #008000; text-decoration-color: #008000\">\"The framework promotes building a stateful graph structure for actions.\"</span>,\n",+ " <span style=\"color: #008000; text-decoration-color: #008000\">\"Use local tracking to effortlessly monitor and interact with your agent's state.\"</span>\n",+ " <span style=\"font-weight: bold\">]</span>,\n",+ " <span style=\"color: #000080; text-decoration-color: #000080; font-weight: bold\">\"youtube_url\"</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">null</span>\n",+ "<span style=\"font-weight: bold\">}</span>\n",+ "</pre>\n"],"text/plain": [- "<graphviz.graphs.Digraph at 0x7f94c0175180>"+ "\u001b[1m{\u001b[0m\n",+ " \u001b[1;34m\"topic\"\u001b[0m: \u001b[32m\"Burr Framework Overview\"\u001b[0m,\n",+ " \u001b[1;34m\"hook\"\u001b[0m: \u001b[32m\"Ever faced challenges while debugging AI applications? Here's a solution!\"\u001b[0m,\n",+ " \u001b[1;34m\"body\"\u001b[0m: \u001b[32m\"Dive into the world of agent applications with Burr! In this quick overview, we explore how Burr helps you debug failing AI calls and track state effectively. Learn to build a graph that connects actions and states, allowing you to resume your application exactly where you left off. Whether it’s fixing an error mid-run or replaying past actions, Burr's observability features enhance your development workflow. Ready to optimize your debugging process? Check out the full video to unravel the potential of Burr!\"\u001b[0m,\n",+ " \u001b[1;34m\"concepts\"\u001b[0m: \u001b[1m[\u001b[0m\n",+ " \u001b[1m{\u001b[0m\n",+ " \u001b[1;34m\"term\"\u001b[0m: \u001b[32m\"Agent Application\"\u001b[0m,\n",+ " \u001b[1;34m\"definition\"\u001b[0m: \u001b[32m\"A system that models states and actions to create decision-making processes.\"\u001b[0m,\n",+ " \u001b[1;34m\"timestamp\"\u001b[0m: \u001b[1;36m105.479\u001b[0m\n",+ " \u001b[1m}\u001b[0m,\n",+ " \u001b[1m{\u001b[0m\n",+ " \u001b[1;34m\"term\"\u001b[0m: \u001b[32m\"State Object\"\u001b[0m,\n",+ " \u001b[1;34m\"definition\"\u001b[0m: \u001b[32m\"An object that holds the state information for actions to read and write during execution.\"\u001b[0m,\n",+ " \u001b[1;34m\"timestamp\"\u001b[0m: \u001b[1;36m145.56\u001b[0m\n",+ " \u001b[1m}\u001b[0m,\n",+ " \u001b[1m{\u001b[0m\n",+ " \u001b[1;34m\"term\"\u001b[0m: \u001b[32m\"Graph Representation\"\u001b[0m,\n",+ " \u001b[1;34m\"definition\"\u001b[0m: \u001b[32m\"A structural representation of actions and their interconnections in an agent system, depicted as nodes and edges.\"\u001b[0m,\n",+ " \u001b[1;34m\"timestamp\"\u001b[0m: \u001b[1;36m179.28\u001b[0m\n",+ " \u001b[1m}\u001b[0m\n",+ " \u001b[1m]\u001b[0m,\n",+ " \u001b[1;34m\"key_takeaways\"\u001b[0m: \u001b[1m[\u001b[0m\n",+ " \u001b[32m\"Burr allows near-instantaneous debugging without restarting from scratch.\"\u001b[0m,\n",+ " \u001b[32m\"The framework promotes building a stateful graph structure for actions.\"\u001b[0m,\n",+ " \u001b[32m\"Use local tracking to effortlessly monitor and interact with your agent's state.\"\u001b[0m\n",+ " \u001b[1m]\u001b[0m,\n",+ " \u001b[1;34m\"youtube_url\"\u001b[0m: \u001b[3;35mnull\u001b[0m\n",+ "\u001b[1m}\u001b[0m\n"]},- "execution_count": 22,"metadata": {},- "output_type": "execute_result"+ "output_type": "display_data"}],"source": [- "from burr.core import ApplicationBuilder\n",- "from burr.core.persistence import SQLLitePersister\n",- "\n",- "persister = SQLLitePersister(db_path=\".burr.db\", table_name=\"burr_state\")\n",- "persister.initialize() # this will create the db and table\n",- "\n",- "application = (\n",- " ApplicationBuilder()\n",- " .with_actions(\n",- " get_youtube_transcript,\n",- " generate_post.bind(llm_client=llm_client),\n",- " rewrite_post.bind(llm_client=llm_client),\n",- " )\n",- " .with_transitions(\n",- " (\"get_youtube_transcript\", \"generate_post\"),\n",- " (\"generate_post\", \"rewrite_post\"),\n",- " (\"rewrite_post\", \"rewrite_post\"),\n",- " )\n",- " .with_state_persister(persister)\n",- " .with_entrypoint(\"get_youtube_transcript\")\n",- " .with_tracker(project=\"youtube-post\")\n",- " .build()\n",- ")\n",- "application.visualize()"- ]- },- {- "cell_type": "code",- "execution_count": 23,- "metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "LAST ACTION : generate_post: transcript -> post \n",- "\n",- "TOPIC: Enhancing AI Agent Applications with B and Hamilton\n",- "\n",- "🚀 Dive into the world of AI agents and discover how to streamline your debugging process with B!\n",- "\n",- "Are you tired of constant debugging cycles in your AI projects? In our latest video, we explore B, a framework designed to help you monitor and debug agent applications efficiently. Learn how B allows you to visualize your application’s state, manage complex action flows, and even fork your state at any point to tackle bugs without restarting from scratch. Plus, discover its sister framework, Hamilton, perfect for managing data pipelines! Curious to learn how to simplify your debugging and enhance your application development? 🎥 Watch the full video now!\n",- "\n",- "Learn to visualize and debug agent applications effectively with B. Understand how to manage states and actions using a graphical approach. Explore the integration of Hamilton for complex data pipeline processes.\n",- "\n",- "link: https://www.youtube.com/watch?v=hqutVJyd3TI\n",- "\n",- "CONCEPTS\n",- "0:1 - B Framework: A framework for building and debugging AI agent applications, allowing for state tracking and action visualization.\n",- "2:25 - State Object: An object that retains the state of an application, enabling actions to read and write within that context.\n",- "2:0 - Graph Visualization: A depiction of actions and states in a flowchart-like manner to streamline application behavior and debugging.\n"- ]- }- ],- "source": [- "# this will run \n",- "last_action, result, state = application.run(\n",+ "_, streaming_container = app.stream_result(\n"," halt_after=[\"generate_post\"],\n"," inputs={\"youtube_url\": \"https://www.youtube.com/watch?v=hqutVJyd3TI\"},\n",")\n",- "print(\"LAST ACTION :\", last_action, \"\\n\")\n",- "print(state[\"post\"].display())"- ]- },- {- "cell_type": "code",- "execution_count": 24,- "metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "LAST ACTION : rewrite_post: post -> post \n",- "\n",- "TOPIC: Enhancing AI Agent Applications with B and Hamilton\n",- "\n",- "🚀 Dive into the world of AI agents and discover how to streamline your debugging process with B!\n",- "\n",- "Are you encountering challenges with debugging cycles in your AI projects? In our latest video, we explore B, a framework designed for monitoring and debugging agent applications effectively. With B, you can visualize your application’s state, manage complex action flows, and fork your state to tackle bugs without having to restart from scratch. Additionally, we introduce Hamilton, a framework that complements B by simplifying the management of data pipelines. If you are interested in improving your debugging process and application development, we invite you to watch the full video!\n",- "\n",- "Learn to visualize and debug agent applications effectively with B. Understand how to manage states and actions using a graphical approach. Explore the integration of Hamilton for complex data pipeline processes.\n",- "\n",- "link: https://www.youtube.com/watch?v=hqutVJyd3TI\n",- "\n",- "CONCEPTS\n",- "0:1 - B Framework: A framework for building and debugging AI agent applications, allowing for state tracking and action visualization.\n",- "2:25 - State Object: An object that retains the state of an application, enabling actions to read and write within that context.\n",- "2:0 - Graph Visualization: A depiction of actions and states in a flowchart-like manner to streamline application behavior and debugging.\n"- ]- }- ],- "source": [- "last_action, result, state = application.step(\n",- " inputs={\"user_prompt\": \"Adopt a professional tone that avoids incredible claims. Stay close to the facts, but demonstrate enthusiasm\"},\n",- ")\n",- "\n",- "print(\"LAST ACTION :\", last_action, \"\\n\")\n",- "print(state[\"post\"].display())"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "Then, you can retrieve the store app ids and resume from there. This requires adding a `.initialize_from()` with the `persister`. Also, we're getting the `app_id` from storage via `persister.list_app_ids()`. By passing it to the `.with_identifiers(app_id=...)`, this \"reloaded application\" will logged and tracked in succession of the original app (as you can see in the Burr UI).\n",- "\n",- "Below, we're building a new `Application` from the persisted state. When we print the `State`, we're able to retrieve the previously generated (you can see the matching titles)!"- ]- },- {- "cell_type": "code",- "execution_count": 36,- "metadata": {},- "outputs": [- {- "name": "stdout",- "output_type": "stream",- "text": [- "Enhancing AI Agent Applications with B and Hamilton\n"- ]- }- ],- "source": [- "app_id = persister.list_app_ids(\"\")[0]\n",- "\n",- "reloaded_app = (\n",- " ApplicationBuilder()\n",- " .with_actions(\n",- " get_youtube_transcript,\n",- " generate_post.bind(llm_client=llm_client),\n",- " rewrite_post.bind(llm_client=llm_client),\n",- " )\n",- " .with_transitions(\n",- " (\"get_youtube_transcript\", \"generate_post\"),\n",- " (\"generate_post\", \"rewrite_post\"),\n",- " (\"rewrite_post\", \"rewrite_post\"),\n",- " )\n",- " .initialize_from(\n",- " persister,\n",- " resume_at_next_action=True,\n",- " default_state={},\n",- " default_entrypoint=\"get_youtube_transcript\",\n",- " )\n",- " .with_state_persister(persister)\n",- " .with_identifiers(app_id=app_id)\n",- " .with_tracker(project=\"youtube-post\")\n",- " .build()\n",- ")\n",- "\n",- "print(reloaded_app.state[\"post\"].topic)"- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "Persistence is great for iterative development such as tuning your [Instructor model with validators and constraints](https://python.useinstructor.com/concepts/reask_validation/), but it's also a powerful tool for building [test cases and guard rails](https://burr.dagworks.io/examples/guardrails/creating_tests/). The CLI command `burr-test-case` can generate a `pytest.fixture` to resume your app from a given state."- ]- },- {- "cell_type": "markdown",- "metadata": {},- "source": [- "### 3. Portability\n",- "\n",- "In the GitHub repository, you can find the same Burr `Application` defined in `application.py`, which can be executed via `python application.py`. Also, we provide a boilerplate FastAPI application in `server.py` which imports the `Application` defined in `application.py`"+ "for post in streaming_container:\n",+ " assert isinstance(post, SocialMediaPost)\n",+ " clear_output(wait=True)\n",+ " obj = post.model_dump()\n",+ " json_str = json.dumps(obj, indent=2)\n",+ " print_json(json_str)"]}],@@ -828,9 +470,9 @@"name": "python","nbconvert_exporter": "python","pygments_lexer": "ipython3",- "version": "3.10.9"+ "version": "3.12.0"}},"nbformat": 4,- "nbformat_minor": 4+ "nbformat_minor": 5}