Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.
Python aiburrchatbot-frameworkdagsgenerative-aigraphshacktoberfestllmopsllmsmlopspersistent-data-structurestate-machinestate-managementvisibility
Adds documentation for recursive tracking
elijahbenizzy957a6f39e540
diff --git a/docs/_static/meme.png b/docs/_static/meme.pngnew file mode 100644index 00000000..c66b800bBinary files /dev/null and b/docs/_static/meme.png differdiff --git a/docs/_static/recursive_steps.png b/docs/_static/recursive_steps.pngnew file mode 100644index 00000000..c8313d2eBinary files /dev/null and b/docs/_static/recursive_steps.png differdiff --git a/docs/concepts/index.rst b/docs/concepts/index.rstindex 2dec632c..a054e884 100644--- a/docs/concepts/index.rst+++ b/docs/concepts/index.rst@@ -19,4 +19,5 @@ Overview of the concepts -- read these to get a mental model for how Burr works.streaming-actionshooksadditional-visibility+ recursionplanned-capabilitiesdiff --git a/docs/concepts/recursion.rst b/docs/concepts/recursion.rstnew file mode 100644index 00000000..d15884e9--- /dev/null+++ b/docs/concepts/recursion.rst@@ -0,0 +1,121 @@+======================+Recursive Applications+======================++Burr supports sub-applications. E.G. running Burr within Burr.+Currently this capability is only done with tracking (linking applications),+but we are actively working on building first-level constructs that allow this.++This can enable the following use-cases (among many others):++1. Parallel results to different LLMs for comparison+2. Multi simultaneous tasks/chains for an LLM+3. Black-boxing an action as a subset of tasks+4. Running multiple queries simultaneously and getting an LLM to synthesize.++And so on. These can be arbitrarily complex and nested, allowing agents consisting of agents consisting of agents...++.. image:: ../_static/meme.png+ :align: center++You can find an example `here <https://github.com/dagworks-inc/burr/tree/main/examples/recursive>`_.++--------+Tracking+--------++Currently, to leverage nested Burr apps, you do the following:++1. Create a parent app (as you normally would)+2. Leverage tracking+3. Create an action inside that+4. Run your Burr sub-application inside that action+5. Wire through tracker + parent data for the sub-application+++After creating a simple application (1)/(2) above, you can wire through the tracker like this (some pseudocode, see the example for a fully working application).+++.. code-block:: python++ @action(reads=["parameters_to_map"], writes=["joined_results"])+ def run_multiple_sub_apps(state: State, __context: ApplicationContext) -> Tuple[dict, State]:+ # for every parameter incoming, run a sub-application+ results = []+ for parameter_set in state["parameters_to_map"]:+ subapp = (+ ApplicationBuilder()+ .with_actions(...)+ .with_transitions(...)+ .with_tracker(__context.tracker.copy())+ .with_spawning_parent(+ __context.app_id,+ __context.sequence_id,+ __context.partition_key,+ )+ .with_entrypoint(...)+ .with_state(...)+ .build()+ )+ results.append(subapp.run(...))++ joined = _join(results)++ return {"joined_results": joined}, state.update(joined_results=joined)++Note we're simply passing through the tracker to the next app, which sets a link between them.+The tracker knows how to record the link so its in the UI.++You can also run this in parallel -- this leverages `joblib <https://joblib.readthedocs.io/en/latest/>`_'s+Parallel and delayed functions, but you can use whatever tooling you want (multiprocessing, threading, etc...).++.. code-block:: python++ @action(reads=["parameters_to_map"], writes=["joined_results"])+ def run_multiple_sub_apps(state: State, __context: ApplicationContext) -> Tuple[dict, State]:+ def run_subapp(parameter_set):+ subapp = (+ ApplicationBuilder()+ .with_actions(...)+ .with_transitions(...)+ .with_tracker(__context.tracker.copy())+ .with_spawning_parent(+ __context.app_id,+ __context.sequence_id,+ __context.partition_key,+ )+ .with_entrypoint(...)+ .with_state(...)+ .build()+ )+ return subapp.run(...)++ results = Parallel(n_jobs=-1)(delayed(run_subapp)(parameter_set) for parameter_set in state["parameters_to_map"])+ joined = _join(results)++ return {"joined_results": joined}, state.update(joined_results=joined)++Note the following:++1. You don't have to pass in partition_key, but you need to pass in the app_id and sequence_id so the app can track+2. You currently need to run ``__context.tracker.copy()`` to ensure the tracker is passed through correctly. We will likely be relaxing this, but it could corrupt the tracking state if you forget.+3. You may want to have more control over the application ID's -- you may want to use a stable hash of the parameters and the parent App ID++For persistence, you'll want to use (3) to ensure a stable hash -- have the sub-application load its own state from the ID, resuming where it left off.++When you track in the UI, you will see the following (in this example we're generating poems of different styles in parallel):++.. image:: ../_static/recursive_steps.png+ :align: center++-------------------+Future Improvements+-------------------++We are working on the following:++1. A recursive application to allow for more ergonomic management of the above+2. Static visualization of the recursive application when using (1)+3. A more ergonomic way to pass through the tracker++Stay tuned!diff --git a/examples/recursive/README.md b/examples/recursive/README.mdnew file mode 100644index 00000000..81348802--- /dev/null+++ b/examples/recursive/README.md@@ -0,0 +1,40 @@+# Recursive Applications++Burr supports running applications inside applications. Currently this is done through sharing links in tracking,+but we will be adding more features to make this more ergonomic.++## Example Structure++This example runs a simple applications. We:+1. Gather input from the user for a poem subject, a maximum number of drafts, and a list of poem types in parallel+2. Run a recursive application that generates a poem for each poem type+3. Display the generated poems back to the user++The overall application looks like this:++++The `generate_all_poems` then runs `n` (one for each poem type) instances of the sub-application:++++## Running the example++To run this, you'll need an OpenAI key set.+Then you can run:++```bash+export OPENAI_API_KEY=...+python application.py+```++This will generate three poems about state machines, one for each type of poem, then combine at the end.++To change the poem type, edit `application.py` and change the `poem_types` list in the mainline.++Run the example with `python application.py`. Then, ensure `burr` is running, and navigate to the+UI: [http://localhost:7241/project/demo:parallelism_poem_generation](http://localhost:7241/project/demo:parallelism_poem_generation).+++FOr more information, read the documentation on [recursive applications](https://burr.dagworks.io/concepts/recursion).+```diff --git a/examples/recursive/application.py b/examples/recursive/application.pynew file mode 100644index 00000000..1e53b9f7--- /dev/null+++ b/examples/recursive/application.py@@ -0,0 +1,213 @@+from typing import List, Tuple++import joblib+import openai+from joblib import parallel_config++from burr.core import Application, ApplicationBuilder, Condition, State, action+from burr.core.application import ApplicationContext+++# full agent+def _query_llm(prompt: str) -> str:+ """Simple wrapper around the OpenAI API."""+ client = openai.Client()+ return (+ client.chat.completions.create(+ model="gpt-4o",+ messages=[+ {"role": "system", "content": "You are a helpful assistant"},+ {"role": "user", "content": prompt},+ ],+ )+ .choices[0]+ .message.content+ )+++@action(+ reads=["feedback", "current_draft", "poem_type", "prompt"],+ writes=["current_draft", "draft_history", "num_drafts"],+)+def write(state: State) -> Tuple[dict, State]:+ """Writes a draft of a poem."""+ poem_subject = state["prompt"]+ poem_type = state["poem_type"]+ current_draft = state.get("current_draft")+ feedback = state.get("feedback")++ parts = [+ f'You are an AI poet. Create a {poem_type} poem on the following subject: "{poem_subject}".'+ ]++ if current_draft:+ parts.append(f'Here is the current draft of the poem: "{current_draft}".')++ if feedback:+ parts.append(f'Please incorporate the following feedback: "{feedback}".')++ parts.append(+ f"Ensure the poem is creative, adheres to the style of a {poem_type}, and improves upon the previous draft."+ )++ prompt = "\n".join(parts)++ draft = _query_llm(prompt)++ return {"draft": draft}, state.update(+ current_draft=draft,+ draft_history=state.get("draft_history", []) + [draft],+ ).increment(num_drafts=1)+++@action(reads=["current_draft", "poem_type", "prompt"], writes=["feedback"])+def edit(state: State) -> Tuple[dict, State]:+ """Edits a draft of a poem, providing feedback"""+ poem_subject = state["prompt"]+ poem_type = state["poem_type"]+ current_draft = state["current_draft"]++ prompt = f"""+ You are an AI poetry critic. Review the following {poem_type} poem based on the subject: "{poem_subject}".+ Here is the current draft of the poem: "{current_draft}".+ Provide detailed feedback to improve the poem. If the poem is already excellent and needs no changes, simply respond with an empty string.+ """++ feedback = _query_llm(prompt)++ return {"feedback": feedback}, state.update(feedback=feedback)+++@action(reads=["current_draft"], writes=["final_draft"])+def final_draft(state: State) -> Tuple[dict, State]:+ return {"final_draft": state["current_draft"]}, state.update(final_draft=state["current_draft"])+++def _create_sub_application(+ max_num_drafts: int,+ spawning_application_context: ApplicationContext,+ poem_type: str,+ prompt: str,+) -> Application:+ """Utility to create sub-application -- note"""+ out = (+ ApplicationBuilder()+ .with_actions(+ edit,+ write,+ final_draft,+ )+ .with_transitions(+ ("write", "edit", Condition.expr(f"num_drafts < {max_num_drafts}")),+ ("write", "final_draft"),+ ("edit", "final_draft", Condition.expr("len(feedback) == 0")),+ ("edit", "write"),+ )+ .with_tracker(spawning_application_context.tracker.copy()) # remember to do `copy()` here!+ .with_spawning_parent(+ spawning_application_context.app_id,+ spawning_application_context.sequence_id,+ spawning_application_context.partition_key,+ )+ .with_entrypoint("write")+ .with_state(+ current_draft=None,+ poem_type=poem_type,+ prompt=prompt,+ feedback=None,+ )+ .build()+ )+ return out+++# full agent+@action(+ reads=[],+ writes=[+ "max_drafts",+ "poem_types",+ "poem_subject",+ ],+)+def user_input(+ state: State, max_drafts: int, poem_types: List[str], poem_subject: str+) -> Tuple[dict, State]:+ """Collects user input for the poem generation process."""+ return {+ "max_drafts": max_drafts,+ "poem_types": poem_types,+ "poem_subject": poem_subject,+ }, state.update(max_drafts=max_drafts, poem_types=poem_types, poem_subject=poem_subject)+++@action(reads=["max_drafts", "poem_types", "poem_subject"], writes=["proposals"])+def generate_all_poems(state: State, __context: ApplicationContext) -> Tuple[dict, State]:+ # create one each+ apps = [+ _create_sub_application(state["max_drafts"], __context, poem_type, state["poem_subject"])+ for poem_type in state["poem_types"]+ ]+ # run them all in parallel+ with parallel_config(backend="threading", n_jobs=3):+ all_results = joblib.Parallel()(+ joblib.delayed(app.run)(halt_after=["final_draft"])+ for app, poem_type in zip(apps, state["poem_types"])+ )+ proposals = []+ for *_, substate in all_results:+ proposals.append(substate["final_draft"])++ return {"proposals": proposals}, state.update(proposals=proposals)+++@action(reads=["proposals", "prompts"], writes=["final_results"])+def final_results(state: State) -> Tuple[dict, State]:+ # joins them into a string+ proposals = state["proposals"]+ final_results = "\n\n".join(+ [f"{poem_type}:\n{proposal}" for poem_type, proposal in zip(state["poem_types"], proposals)]+ )+ return {"final_results": final_results}, state.update(final_results=final_results)+++def application() -> Application:+ return (+ ApplicationBuilder()+ .with_actions(+ user_input,+ generate_all_poems,+ final_results,+ )+ .with_transitions(+ ("user_input", "generate_all_poems"),+ ("generate_all_poems", "final_results"),+ )+ .with_tracker(project="demo:parallelism_poem_generation")+ .with_entrypoint("user_input")+ .build()+ )+++if __name__ == "__main__":+ app = application()+ app.visualize(output_file_path="statemachine", format="png")+ _create_sub_application(+ 2,+ app.context,+ "sonnet",+ "state machines",+ ).visualize(output_file_path="statemachine_sub", format="png")+ app.run(+ halt_after=["final_results"],+ inputs={+ "max_drafts": 2,+ "poem_types": [+ "sonnet",+ "limerick",+ "haiku",+ "acrostic",+ ],+ "poem_subject": "state machines",+ },+ )diff --git a/examples/recursive/requirements.txt b/examples/recursive/requirements.txtnew file mode 100644index 00000000..cb387dcd--- /dev/null+++ b/examples/recursive/requirements.txt@@ -0,0 +1,2 @@+burr[learn]+openaidiff --git a/examples/recursive/statemachine.png b/examples/recursive/statemachine.pngnew file mode 100644index 00000000..8adb0b82Binary files /dev/null and b/examples/recursive/statemachine.png differdiff --git a/examples/recursive/statemachine_sub.png b/examples/recursive/statemachine_sub.pngnew file mode 100644index 00000000..6328886eBinary files /dev/null and b/examples/recursive/statemachine_sub.png differ