Skip to content

armgit / burr

Public Imported from GitHub
0

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

Adds parallelism example - WIP but we want to get out before the Show HN tomorrow

elijahbenizzy5f3df0bfd4f9
diff --git a/examples/parallelism/README.md b/examples/parallelism/README.md
new file mode 100644
index 00000000..87cb8972
--- /dev/null
+++ b/examples/parallelism/README.md
@@ -0,0 +1,7 @@
+# Parallelism
+
+In this example we go over Burr's parallelism capabilities. It is based on the documentation (https://burr.dagworks.io/pull/370/concepts/parallelism/), demonstrating the `MapStates` capabilities.
+
+See [the notebook](./notebook.ipynb) for the full example.
+
+You can follow along with this [Loom video](https://www.loom.com/share/a871abe84f974a499d7fb5bae42c35ae)
diff --git a/examples/parallelism/notebook.ipynb b/examples/parallelism/notebook.ipynb
new file mode 100644
index 00000000..b727bf64
--- /dev/null
+++ b/examples/parallelism/notebook.ipynb
@@ -0,0 +1,632 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7a4c338b-90dc-4866-9cc9-5c9af38f9537",
+ "metadata": {},
+ "source": [
+ "# Parallelism\n",
+ "\n",
+ "In this notebook we're going to go over how to run parallel sub-applications, using one of the examples from [the documentation](https://burr.dagworks.io/concepts/parallelism/).\n",
+ "\n",
+ "We will be showing how to write a simple Burr application that compares models for the same prompt. We will go through the following, showing how Parallelism simplifies it:\n",
+ "\n",
+ "1. Creating recursive sub-applications yourself\n",
+ "2. Wiring through tracking information with [recursive tracking capabilities](https://burr.dagworks.io/concepts/recursion/)\n",
+ "3. Using the map/reduce capabilities with Burr's [parallelism tooling](https://burr.dagworks.io/concepts/parallelism/)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "10459bdd-2154-43c6-84d2-7bee4825ab9b",
+ "metadata": {},
+ "source": [
+ "# Imports"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "id": "e95aa6fe-1eaa-490d-b7fd-ff956306bcc5",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from burr.core import action, State, ApplicationBuilder, ApplicationContext\n",
+ "from burr.core.parallelism import MapStates, RunnableGraph\n",
+ "from typing import Callable, Generator, List, Dict, Any\n",
+ "\n",
+ "from IPython.display import HTML, IFrame\n",
+ "\n",
+ "\n",
+ "import pprint\n",
+ "\n",
+ "import openai\n",
+ "\n",
+ "\n",
+ "from concurrent.futures import ThreadPoolExecutor"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "id": "e59f1418-68f8-47c2-8034-4b928211acbe",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "The burr.integrations.notebook extension is already loaded. To reload it, use:\n",
+ " %reload_ext burr.integrations.notebook\n",
+ "Burr UI already running at http://127.0.0.1:7241\n"
+ ]
+ }
+ ],
+ "source": [
+ "%load_ext burr.integrations.notebook\n",
+ "%burr_ui"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "948a51e7-75e8-4d47-a23f-c24a89e8be60",
+ "metadata": {},
+ "source": [
+ "# Helper/Shared Functions\n",
+ "\n",
+ "In this we'll set up the application -- creating our action functions as well as a few different helper functions we'll want to use.\n",
+ "\n",
+ "The graph will have a simple (3-node) structure:\n",
+ "\n",
+ "1. `process_input` -- pulls data in from the user at runtime\n",
+ "2. `run_llms` -- runs the different models in parallel\n",
+ "3. `join_outputs` -- writes a map of LLM -> result\n",
+ "\n",
+ "We will first define nodes (1) and (3), then redefine (2) according to the approaches above."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "id": "0998576a-c8cb-45a2-94ce-d8ce326e3be7",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "MODELS = [\n",
+ " \"gpt-4\",\n",
+ " \"gpt-4-turbo\",\n",
+ " \"gpt-3.5-turbo\",\n",
+ " \"gpt-3.5-turbo-16k\",\n",
+ " \"text-davinci-003\",\n",
+ " \"text-davinci-002\",\n",
+ "]\n",
+ "\n",
+ "def _query_llm(model: str, prompt: str) -> str:\n",
+ " \"\"\"Simple function to query our LLM -- we'll only support OpenAI for now.\"\"\"\n",
+ " \n",
+ " if model not in MODELS:\n",
+ " raise ValueError(f\"Model '{model}' is not an OpenAI-supported model.\")\n",
+ "\n",
+ " client = openai.Client()\n",
+ " \n",
+ " response = client.chat.completions.create(\n",
+ " model=model,\n",
+ " messages=[{\"role\": \"user\", \"content\": prompt}]\n",
+ " )\n",
+ " return response.choices[0].message.content\n",
+ "\n",
+ "\n",
+ "@action(reads=[], writes=[\"prompt\", \"models\"])\n",
+ "def process_input(state: State, prompt: str, models: List[str]) -> State:\n",
+ " \"\"\"First node in our graph -- will take the prompt input and write to state\"\"\"\n",
+ " return state.update(prompt=prompt, models=models)\n",
+ "\n",
+ "\n",
+ "@action(reads=[\"responses\", \"models\"], writes=[\"all_responses\"])\n",
+ "def join_outputs(state: State) -> State:\n",
+ " \"\"\"Final node, just joins in a dictionary\"\"\"\n",
+ " joined_results = {}\n",
+ " for response, model in zip(state[\"responses\"], state[\"models\"]):\n",
+ " joined_results[model] = response\n",
+ " return state.update(all_responses=joined_results)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "971e9a62-543a-4866-854c-60bbe89039ec",
+ "metadata": {},
+ "source": [
+ "# Approach (1) -- doing this manually\n",
+ "\n",
+ "Nothing stops you from running multiple queries in the same action. In this we're going to just define the middle action to run in parallel -- it will launch an application in Burr, and wait for the results.\n",
+ "\n",
+ "This is the \"manual\" approach -- how you might achieve parallelism without Burr. In it, we're going to do the following:\n",
+ "\n",
+ "1. Create a sub-application with a single node that queries the LLM\n",
+ "2. Run multiple variants of it in parallel\n",
+ "3. Join the result\n",
+ "\n",
+ "This is a useful approach as it is simple and looks like standard python code. That said, it lacks in visibility -- you have no way to know what the sub-LLM calls are doing.\n",
+ "\n",
+ "First, we'll define the middle action. Next, we'll define the application. Then we can run!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "id": "27ee761e-f7e8-4da7-82f2-349653cc6a7e",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/svg+xml": [
+ "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
+ "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
+ " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
+ "<!-- Generated by graphviz version 12.0.0 (20240704.0754)\n",
+ " -->\n",
+ "<!-- Pages: 1 -->\n",
+ "<svg width=\"230pt\" height=\"241pt\"\n",
+ " viewBox=\"0.00 0.00 229.85 241.40\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
+ "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 237.4)\">\n",
+ "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-237.4 225.85,-237.4 225.85,4 -4,4\"/>\n",
+ "<!-- process_input -->\n",
+ "<g id=\"node1\" class=\"node\">\n",
+ "<title>process_input</title>\n",
+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M152.1,-167.8C152.1,-167.8 70.5,-167.8 70.5,-167.8 64.5,-167.8 58.5,-161.8 58.5,-155.8 58.5,-155.8 58.5,-143.2 58.5,-143.2 58.5,-137.2 64.5,-131.2 70.5,-131.2 70.5,-131.2 152.1,-131.2 152.1,-131.2 158.1,-131.2 164.1,-137.2 164.1,-143.2 164.1,-143.2 164.1,-155.8 164.1,-155.8 164.1,-161.8 158.1,-167.8 152.1,-167.8\"/>\n",
+ "<text text-anchor=\"middle\" x=\"111.3\" y=\"-143.7\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">process_input</text>\n",
+ "</g>\n",
+ "<!-- query_multiple_models -->\n",
+ "<g id=\"node4\" class=\"node\">\n",
+ "<title>query_multiple_models</title>\n",
+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M180.22,-102.2C180.22,-102.2 42.38,-102.2 42.38,-102.2 36.38,-102.2 30.38,-96.2 30.38,-90.2 30.38,-90.2 30.38,-77.6 30.38,-77.6 30.38,-71.6 36.38,-65.6 42.38,-65.6 42.38,-65.6 180.22,-65.6 180.22,-65.6 186.22,-65.6 192.22,-71.6 192.22,-77.6 192.22,-77.6 192.22,-90.2 192.22,-90.2 192.22,-96.2 186.22,-102.2 180.22,-102.2\"/>\n",
+ "<text text-anchor=\"middle\" x=\"111.3\" y=\"-78.1\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">query_multiple_models</text>\n",
+ "</g>\n",
+ "<!-- process_input&#45;&gt;query_multiple_models -->\n",
+ "<g id=\"edge3\" class=\"edge\">\n",
+ "<title>process_input&#45;&gt;query_multiple_models</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M111.3,-130.78C111.3,-125.55 111.3,-119.67 111.3,-113.92\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"114.8,-114.03 111.3,-104.03 107.8,-114.03 114.8,-114.03\"/>\n",
+ "</g>\n",
+ "<!-- input__models -->\n",
+ "<g id=\"node2\" class=\"node\">\n",
+ "<title>input__models</title>\n",
+ "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"102.6,-233.4 0,-233.4 0,-196.8 102.6,-196.8 102.6,-233.4\"/>\n",
+ "<text text-anchor=\"middle\" x=\"51.3\" y=\"-209.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">input: models</text>\n",
+ "</g>\n",
+ "<!-- input__models&#45;&gt;process_input -->\n",
+ "<g id=\"edge1\" class=\"edge\">\n",
+ "<title>input__models&#45;&gt;process_input</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M68.01,-196.38C73.86,-190.18 80.56,-183.08 86.91,-176.35\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"89.32,-178.9 93.64,-169.22 84.23,-174.09 89.32,-178.9\"/>\n",
+ "</g>\n",
+ "<!-- input__prompt -->\n",
+ "<g id=\"node3\" class=\"node\">\n",
+ "<title>input__prompt</title>\n",
+ "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"221.85,-233.4 120.75,-233.4 120.75,-196.8 221.85,-196.8 221.85,-233.4\"/>\n",
+ "<text text-anchor=\"middle\" x=\"171.3\" y=\"-209.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">input: prompt</text>\n",
+ "</g>\n",
+ "<!-- input__prompt&#45;&gt;process_input -->\n",
+ "<g id=\"edge2\" class=\"edge\">\n",
+ "<title>input__prompt&#45;&gt;process_input</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M154.59,-196.38C148.74,-190.18 142.04,-183.08 135.69,-176.35\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"138.37,-174.09 128.96,-169.22 133.28,-178.9 138.37,-174.09\"/>\n",
+ "</g>\n",
+ "<!-- join_outputs -->\n",
+ "<g id=\"node5\" class=\"node\">\n",
+ "<title>join_outputs</title>\n",
+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M146.47,-36.6C146.47,-36.6 76.12,-36.6 76.12,-36.6 70.12,-36.6 64.12,-30.6 64.12,-24.6 64.12,-24.6 64.12,-12 64.12,-12 64.12,-6 70.12,0 76.12,0 76.12,0 146.47,0 146.47,0 152.47,0 158.47,-6 158.47,-12 158.47,-12 158.47,-24.6 158.47,-24.6 158.47,-30.6 152.47,-36.6 146.47,-36.6\"/>\n",
+ "<text text-anchor=\"middle\" x=\"111.3\" y=\"-12.5\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">join_outputs</text>\n",
+ "</g>\n",
+ "<!-- query_multiple_models&#45;&gt;join_outputs -->\n",
+ "<g id=\"edge4\" class=\"edge\">\n",
+ "<title>query_multiple_models&#45;&gt;join_outputs</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M111.3,-65.18C111.3,-59.95 111.3,-54.07 111.3,-48.32\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"114.8,-48.43 111.3,-38.43 107.8,-48.43 114.8,-48.43\"/>\n",
+ "</g>\n",
+ "</g>\n",
+ "</svg>\n"
+ ],
+ "text/plain": [
+ "<burr.core.application.Application at 0x168ab8680>"
+ ]
+ },
+ "execution_count": 18,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "@action(reads=[\"models\", \"prompt\"], writes=[\"responses\"])\n",
+ "def query_multiple_models(state: State) -> State:\n",
+ " \"\"\"Query multiple models in parallel and store the results in the state.\"\"\"\n",
+ " models: List[str] = state[\"models\"]\n",
+ " prompt: str = state[\"prompt\"]\n",
+ " \n",
+ " def query_model(model: str) -> str:\n",
+ " return _query_llm(model, prompt)\n",
+ "\n",
+ " with ThreadPoolExecutor() as executor:\n",
+ " futures = {executor.submit(query_model, model): model for model in models}\n",
+ " results = {}\n",
+ " for future in futures:\n",
+ " model = futures[future]\n",
+ " results[model] = future.result()\n",
+ " \n",
+ " return state.update(responses=results)\n",
+ "\n",
+ "app = (\n",
+ " ApplicationBuilder()\n",
+ " .with_actions(\n",
+ " process_input,\n",
+ " query_multiple_models,\n",
+ " join_outputs\n",
+ " ).with_entrypoint(\"process_input\")\n",
+ " .with_tracker(project=\"demo_parallelism\")\n",
+ " .with_transitions(\n",
+ " (\"process_input\", \"query_multiple_models\"),\n",
+ " (\"query_multiple_models\", \"join_outputs\")\n",
+ " )\n",
+ " .build()\n",
+ ")\n",
+ "app"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "id": "ab7e1041-fd7e-438e-8a78-aa468830e2f6",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "action_, _, state = app.run(\n",
+ " inputs={\"prompt\": \"what is the meaning of life?\", \"models\" : [\"gpt-4\", \"gpt-4-turbo\", \"gpt-3.5-turbo\"]},\n",
+ " halt_after=[\"join_outputs\"]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "id": "c99f7a75-1410-4aa6-b60d-16a592c3ae5f",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " <iframe\n",
+ " width=\"100%\"\n",
+ " height=\"700px\"\n",
+ " src=\"http://localhost:7241/project/demo_parallelism/null/47a810b1-75d4-4a49-8bd3-a24e816aba34\"\n",
+ " frameborder=\"0\"\n",
+ " allowfullscreen\n",
+ " \n",
+ " ></iframe>\n",
+ " "
+ ],
+ "text/plain": [
+ "<IPython.lib.display.IFrame at 0x168fcc770>"
+ ]
+ },
+ "execution_count": 20,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "IFrame(f'http://localhost:7241/project/demo_parallelism/null/{app.uid}', width=\"100%\", height=\"700px\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "becfab1b-f89d-49a0-b1af-2f01359bfefa",
+ "metadata": {},
+ "source": [
+ "# Approach (2) -- using recursion\n",
+ "\n",
+ "Burr allows you to wire through tracking in the UI so you can visualize sub-applications. For this, we will be representing the LLM calls with different models as their own (single-node) application. While it is built for more complex sub-application shapes, it works quite well with the simplicity of a single node app."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "14a89b97-ae8f-4ae7-b1fc-a91182d5658d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "@action(reads=[\"model\", \"prompt\"], writes=[\"response\"])\n",
+ "def inner_query_model(state: State) -> State:\n",
+ " return state.update(response=_query_llm(state[\"model\"], state[\"prompt\"]))\n",
+ " \n",
+ "@action(reads=[\"models\", \"prompt\"], writes=[\"responses\"])\n",
+ "def query_multiple_models(state: State, __context: ApplicationContext) -> State:\n",
+ " \"\"\"Query multiple models in parallel and store the results in the state.\"\"\"\n",
+ " apps = []\n",
+ " for model in state[\"models\"]:\n",
+ " apps.append(\n",
+ " (\n",
+ " model, \n",
+ " (\n",
+ " ApplicationBuilder()\n",
+ " .with_actions(query_model=inner_query_model)\n",
+ " .with_state(model=model, prompt=state[\"prompt\"])\n",
+ " .with_tracker(project=\"demo_parallelism\")\n",
+ " .with_spawning_parent(\n",
+ " app_id=__context.app_id,\n",
+ " sequence_id=__context.sequence_id\n",
+ " )\n",
+ " .with_entrypoint(\"query_model\")\n",
+ " )\n",
+ " .build())\n",
+ " )\n",
+ " \n",
+ " with ThreadPoolExecutor() as executor:\n",
+ " futures = {executor.submit(app.run, halt_after=[\"query_model\"]): model for model, app in apps}\n",
+ " results = {}\n",
+ " for future in futures:\n",
+ " model = futures[future]\n",
+ " results[model] = future.result()\n",
+ " \n",
+ " return state.update(responses=results)\n",
+ "\n",
+ "app_recursion = (\n",
+ " ApplicationBuilder()\n",
+ " .with_actions(\n",
+ " process_input,\n",
+ " query_multiple_models,\n",
+ " join_outputs\n",
+ " ).with_entrypoint(\"process_input\")\n",
+ " .with_tracker(project=\"demo_parallelism\")\n",
+ " .with_transitions(\n",
+ " (\"process_input\", \"query_multiple_models\"),\n",
+ " (\"query_multiple_models\", \"join_outputs\")\n",
+ " )\n",
+ " .build()\n",
+ ")\n",
+ "app_recursion"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ff28d579-084f-463f-9761-b9e85a6d1460",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "action_, _, state = app_recursion.run(\n",
+ " inputs={\"prompt\": \"what is the meaning of life?\", \"models\" : [\"gpt-4\", \"gpt-4-turbo\", \"gpt-3.5-turbo\"]},\n",
+ " halt_after=[\"join_outputs\"]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "adb02469-9dd6-404e-8a9f-0e50fe21123f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "IFrame(f'http://localhost:7241/project/demo_parallelism/null/{app_recursion.uid}', width=\"100%\", height=\"700px\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f8c09d92-045a-4285-840a-f0099fe8b7cf",
+ "metadata": {},
+ "source": [
+ "# Approach (3) -- using parallel constructs\n",
+ "\n",
+ "We can use Burr's parallel construct to make this even easier. We will build a class that does mapping over states -- in this case, the class will create a recursive action for each model in the `models` field, \n",
+ "enabling you to run the models in parallel. Burr's `MapStates` API will create the applications and run them in parallel. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "id": "ce90216e-5470-4bdb-9d32-c9b7c3e64eb3",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/svg+xml": [
+ "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
+ "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
+ " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
+ "<!-- Generated by graphviz version 12.0.0 (20240704.0754)\n",
+ " -->\n",
+ "<!-- Pages: 1 -->\n",
+ "<svg width=\"230pt\" height=\"241pt\"\n",
+ " viewBox=\"0.00 0.00 229.85 241.40\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
+ "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 237.4)\">\n",
+ "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-237.4 225.85,-237.4 225.85,4 -4,4\"/>\n",
+ "<!-- process_input -->\n",
+ "<g id=\"node1\" class=\"node\">\n",
+ "<title>process_input</title>\n",
+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M152.1,-167.8C152.1,-167.8 70.5,-167.8 70.5,-167.8 64.5,-167.8 58.5,-161.8 58.5,-155.8 58.5,-155.8 58.5,-143.2 58.5,-143.2 58.5,-137.2 64.5,-131.2 70.5,-131.2 70.5,-131.2 152.1,-131.2 152.1,-131.2 158.1,-131.2 164.1,-137.2 164.1,-143.2 164.1,-143.2 164.1,-155.8 164.1,-155.8 164.1,-161.8 158.1,-167.8 152.1,-167.8\"/>\n",
+ "<text text-anchor=\"middle\" x=\"111.3\" y=\"-143.7\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">process_input</text>\n",
+ "</g>\n",
+ "<!-- query_multiple_models -->\n",
+ "<g id=\"node5\" class=\"node\">\n",
+ "<title>query_multiple_models</title>\n",
+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M180.22,-102.2C180.22,-102.2 42.38,-102.2 42.38,-102.2 36.38,-102.2 30.38,-96.2 30.38,-90.2 30.38,-90.2 30.38,-77.6 30.38,-77.6 30.38,-71.6 36.38,-65.6 42.38,-65.6 42.38,-65.6 180.22,-65.6 180.22,-65.6 186.22,-65.6 192.22,-71.6 192.22,-77.6 192.22,-77.6 192.22,-90.2 192.22,-90.2 192.22,-96.2 186.22,-102.2 180.22,-102.2\"/>\n",
+ "<text text-anchor=\"middle\" x=\"111.3\" y=\"-78.1\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">query_multiple_models</text>\n",
+ "</g>\n",
+ "<!-- process_input&#45;&gt;query_multiple_models -->\n",
+ "<g id=\"edge3\" class=\"edge\">\n",
+ "<title>process_input&#45;&gt;query_multiple_models</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M111.3,-130.78C111.3,-125.55 111.3,-119.67 111.3,-113.92\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"114.8,-114.03 111.3,-104.03 107.8,-114.03 114.8,-114.03\"/>\n",
+ "</g>\n",
+ "<!-- input__models -->\n",
+ "<g id=\"node2\" class=\"node\">\n",
+ "<title>input__models</title>\n",
+ "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"102.6,-233.4 0,-233.4 0,-196.8 102.6,-196.8 102.6,-233.4\"/>\n",
+ "<text text-anchor=\"middle\" x=\"51.3\" y=\"-209.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">input: models</text>\n",
+ "</g>\n",
+ "<!-- input__models&#45;&gt;process_input -->\n",
+ "<g id=\"edge1\" class=\"edge\">\n",
+ "<title>input__models&#45;&gt;process_input</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M68.01,-196.38C73.86,-190.18 80.56,-183.08 86.91,-176.35\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"89.32,-178.9 93.64,-169.22 84.23,-174.09 89.32,-178.9\"/>\n",
+ "</g>\n",
+ "<!-- input__prompt -->\n",
+ "<g id=\"node3\" class=\"node\">\n",
+ "<title>input__prompt</title>\n",
+ "<polygon fill=\"none\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"221.85,-233.4 120.75,-233.4 120.75,-196.8 221.85,-196.8 221.85,-233.4\"/>\n",
+ "<text text-anchor=\"middle\" x=\"171.3\" y=\"-209.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">input: prompt</text>\n",
+ "</g>\n",
+ "<!-- input__prompt&#45;&gt;process_input -->\n",
+ "<g id=\"edge2\" class=\"edge\">\n",
+ "<title>input__prompt&#45;&gt;process_input</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M154.59,-196.38C148.74,-190.18 142.04,-183.08 135.69,-176.35\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"138.37,-174.09 128.96,-169.22 133.28,-178.9 138.37,-174.09\"/>\n",
+ "</g>\n",
+ "<!-- join_outputs -->\n",
+ "<g id=\"node4\" class=\"node\">\n",
+ "<title>join_outputs</title>\n",
+ "<path fill=\"#b4d8e4\" stroke=\"black\" d=\"M146.47,-36.6C146.47,-36.6 76.12,-36.6 76.12,-36.6 70.12,-36.6 64.12,-30.6 64.12,-24.6 64.12,-24.6 64.12,-12 64.12,-12 64.12,-6 70.12,0 76.12,0 76.12,0 146.47,0 146.47,0 152.47,0 158.47,-6 158.47,-12 158.47,-12 158.47,-24.6 158.47,-24.6 158.47,-30.6 152.47,-36.6 146.47,-36.6\"/>\n",
+ "<text text-anchor=\"middle\" x=\"111.3\" y=\"-12.5\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\">join_outputs</text>\n",
+ "</g>\n",
+ "<!-- query_multiple_models&#45;&gt;join_outputs -->\n",
+ "<g id=\"edge4\" class=\"edge\">\n",
+ "<title>query_multiple_models&#45;&gt;join_outputs</title>\n",
+ "<path fill=\"none\" stroke=\"black\" d=\"M111.3,-65.18C111.3,-59.95 111.3,-54.07 111.3,-48.32\"/>\n",
+ "<polygon fill=\"black\" stroke=\"black\" points=\"114.8,-48.43 111.3,-38.43 107.8,-48.43 114.8,-48.43\"/>\n",
+ "</g>\n",
+ "</g>\n",
+ "</svg>\n"
+ ],
+ "text/plain": [
+ "<burr.core.application.Application at 0x1693a3f50>"
+ ]
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "@action(reads=[\"model\", \"prompt\"], writes=[\"response\"])\n",
+ "def inner_query_model(state: State) -> State:\n",
+ " return state.update(response=_query_llm(state[\"model\"], state[\"prompt\"]))\n",
+ " \n",
+ "class RunOverMultiplePrompts(MapStates):\n",
+ " def states(\n",
+ " self, state: State, context: ApplicationContext, inputs: Dict[str, Any]\n",
+ " ) -> Generator[State, None, None]:\n",
+ " for model in state[\"models\"]:\n",
+ " yield state.update(model=model)\n",
+ "\n",
+ " def action(self, state: State, inputs: Dict[str, Any]) -> Callable:\n",
+ " return inner_query_model\n",
+ "\n",
+ " def reduce(self, state: State, results: Generator[State, None, None]) -> State:\n",
+ " state = state.update(results=[])\n",
+ " responses = {model: output_state[\"response\"] for model, output_state in zip(state[\"models\"], results)}\n",
+ " return state.update(responses=responses)\n",
+ " \n",
+ " @property\n",
+ " def reads(self) -> List[str]:\n",
+ " return [\"models\", \"prompt\"]\n",
+ "\n",
+ " @property\n",
+ " def writes(self) -> List[str]:\n",
+ " return [\"responses\"]\n",
+ "\n",
+ "app_parallelism = (\n",
+ " ApplicationBuilder()\n",
+ " .with_actions(\n",
+ " process_input,\n",
+ " join_outputs,\n",
+ " query_multiple_models=RunOverMultiplePrompts(),\n",
+ " ).with_entrypoint(\"process_input\")\n",
+ " .with_tracker(project=\"demo_parallelism\")\n",
+ " .with_transitions(\n",
+ " (\"process_input\", \"query_multiple_models\"),\n",
+ " (\"query_multiple_models\", \"join_outputs\")\n",
+ " )\n",
+ " .build()\n",
+ ")\n",
+ "app_parallelism"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "id": "816cf0d7-1f89-4145-a5e3-c2878cec12e1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "action_, _, state = app_parallelism.run(\n",
+ " inputs={\"prompt\": \"what is the meaning of life?\", \"models\" : [\"gpt-4\", \"gpt-4-turbo\", \"gpt-3.5-turbo\"]},\n",
+ " halt_after=[\"join_outputs\"]\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "id": "7ece3113-cf60-47b1-b375-735070c70b67",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " <iframe\n",
+ " width=\"100%\"\n",
+ " height=\"700px\"\n",
+ " src=\"http://localhost:7241/project/demo_parallelism/null/2e663638-f715-4799-b1b8-b04aed8e3b3e\"\n",
+ " frameborder=\"0\"\n",
+ " allowfullscreen\n",
+ " \n",
+ " ></iframe>\n",
+ " "
+ ],
+ "text/plain": [
+ "<IPython.lib.display.IFrame at 0x1682186b0>"
+ ]
+ },
+ "execution_count": 23,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "IFrame(f'http://localhost:7241/project/demo_parallelism/null/{app_parallelism.uid}', width=\"100%\", height=\"700px\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c5e31849-5dd6-4e6c-9fd5-0ca3c3e11abe",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.12.0"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/examples/validate_examples.py b/examples/validate_examples.py
index 342cee9c..f46bf41a 100644
--- a/examples/validate_examples.py
+++ b/examples/validate_examples.py
@@ -25,6 +25,7 @@ FILTERLIST = [
"templates",
"deployment",
"talks",
+ "parallelism", # TODO - remove this shortly
]