Skip to content

armgit / burr

Public Imported from GitHub
0

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

Adds documentation for recursive tracking

elijahbenizzy957a6f39e540
diff --git a/docs/_static/meme.png b/docs/_static/meme.png
new file mode 100644
index 00000000..c66b800b
Binary files /dev/null and b/docs/_static/meme.png differ
diff --git a/docs/_static/recursive_steps.png b/docs/_static/recursive_steps.png
new file mode 100644
index 00000000..c8313d2e
Binary files /dev/null and b/docs/_static/recursive_steps.png differ
diff --git a/docs/concepts/index.rst b/docs/concepts/index.rst
index 2dec632c..a054e884 100644
--- a/docs/concepts/index.rst
+++ b/docs/concepts/index.rst
@@ -19,4 +19,5 @@ Overview of the concepts -- read these to get a mental model for how Burr works.
streaming-actions
hooks
additional-visibility
+ recursion
planned-capabilities
diff --git a/docs/concepts/recursion.rst b/docs/concepts/recursion.rst
new file mode 100644
index 00000000..d15884e9
--- /dev/null
+++ b/docs/concepts/recursion.rst
@@ -0,0 +1,121 @@
+======================
+Recursive Applications
+======================
+
+Burr supports sub-applications. E.G. running Burr within Burr.
+Currently this capability is only done with tracking (linking applications),
+but we are actively working on building first-level constructs that allow this.
+
+This can enable the following use-cases (among many others):
+
+1. Parallel results to different LLMs for comparison
+2. Multi simultaneous tasks/chains for an LLM
+3. Black-boxing an action as a subset of tasks
+4. Running multiple queries simultaneously and getting an LLM to synthesize.
+
+And so on. These can be arbitrarily complex and nested, allowing agents consisting of agents consisting of agents...
+
+.. image:: ../_static/meme.png
+ :align: center
+
+You can find an example `here <https://github.com/dagworks-inc/burr/tree/main/examples/recursive>`_.
+
+--------
+Tracking
+--------
+
+Currently, to leverage nested Burr apps, you do the following:
+
+1. Create a parent app (as you normally would)
+2. Leverage tracking
+3. Create an action inside that
+4. Run your Burr sub-application inside that action
+5. Wire through tracker + parent data for the sub-application
+
+
+After creating a simple application (1)/(2) above, you can wire through the tracker like this (some pseudocode, see the example for a fully working application).
+
+
+.. code-block:: python
+
+ @action(reads=["parameters_to_map"], writes=["joined_results"])
+ def run_multiple_sub_apps(state: State, __context: ApplicationContext) -> Tuple[dict, State]:
+ # for every parameter incoming, run a sub-application
+ results = []
+ for parameter_set in state["parameters_to_map"]:
+ subapp = (
+ ApplicationBuilder()
+ .with_actions(...)
+ .with_transitions(...)
+ .with_tracker(__context.tracker.copy())
+ .with_spawning_parent(
+ __context.app_id,
+ __context.sequence_id,
+ __context.partition_key,
+ )
+ .with_entrypoint(...)
+ .with_state(...)
+ .build()
+ )
+ results.append(subapp.run(...))
+
+ joined = _join(results)
+
+ return {"joined_results": joined}, state.update(joined_results=joined)
+
+Note we're simply passing through the tracker to the next app, which sets a link between them.
+The tracker knows how to record the link so its in the UI.
+
+You can also run this in parallel -- this leverages `joblib <https://joblib.readthedocs.io/en/latest/>`_'s
+Parallel and delayed functions, but you can use whatever tooling you want (multiprocessing, threading, etc...).
+
+.. code-block:: python
+
+ @action(reads=["parameters_to_map"], writes=["joined_results"])
+ def run_multiple_sub_apps(state: State, __context: ApplicationContext) -> Tuple[dict, State]:
+ def run_subapp(parameter_set):
+ subapp = (
+ ApplicationBuilder()
+ .with_actions(...)
+ .with_transitions(...)
+ .with_tracker(__context.tracker.copy())
+ .with_spawning_parent(
+ __context.app_id,
+ __context.sequence_id,
+ __context.partition_key,
+ )
+ .with_entrypoint(...)
+ .with_state(...)
+ .build()
+ )
+ return subapp.run(...)
+
+ results = Parallel(n_jobs=-1)(delayed(run_subapp)(parameter_set) for parameter_set in state["parameters_to_map"])
+ joined = _join(results)
+
+ return {"joined_results": joined}, state.update(joined_results=joined)
+
+Note the following:
+
+1. You don't have to pass in partition_key, but you need to pass in the app_id and sequence_id so the app can track
+2. You currently need to run ``__context.tracker.copy()`` to ensure the tracker is passed through correctly. We will likely be relaxing this, but it could corrupt the tracking state if you forget.
+3. You may want to have more control over the application ID's -- you may want to use a stable hash of the parameters and the parent App ID
+
+For persistence, you'll want to use (3) to ensure a stable hash -- have the sub-application load its own state from the ID, resuming where it left off.
+
+When you track in the UI, you will see the following (in this example we're generating poems of different styles in parallel):
+
+.. image:: ../_static/recursive_steps.png
+ :align: center
+
+-------------------
+Future Improvements
+-------------------
+
+We are working on the following:
+
+1. A recursive application to allow for more ergonomic management of the above
+2. Static visualization of the recursive application when using (1)
+3. A more ergonomic way to pass through the tracker
+
+Stay tuned!
diff --git a/examples/recursive/README.md b/examples/recursive/README.md
new file mode 100644
index 00000000..81348802
--- /dev/null
+++ b/examples/recursive/README.md
@@ -0,0 +1,40 @@
+# Recursive Applications
+
+Burr supports running applications inside applications. Currently this is done through sharing links in tracking,
+but we will be adding more features to make this more ergonomic.
+
+## Example Structure
+
+This example runs a simple applications. We:
+1. Gather input from the user for a poem subject, a maximum number of drafts, and a list of poem types in parallel
+2. Run a recursive application that generates a poem for each poem type
+3. Display the generated poems back to the user
+
+The overall application looks like this:
+
+![Recursive Application](statemachine.png)
+
+The `generate_all_poems` then runs `n` (one for each poem type) instances of the sub-application:
+
+![Recursive Application](statemachine_sub.png)
+
+## Running the example
+
+To run this, you'll need an OpenAI key set.
+Then you can run:
+
+```bash
+export OPENAI_API_KEY=...
+python application.py
+```
+
+This will generate three poems about state machines, one for each type of poem, then combine at the end.
+
+To change the poem type, edit `application.py` and change the `poem_types` list in the mainline.
+
+Run the example with `python application.py`. Then, ensure `burr` is running, and navigate to the
+UI: [http://localhost:7241/project/demo:parallelism_poem_generation](http://localhost:7241/project/demo:parallelism_poem_generation).
+
+
+FOr more information, read the documentation on [recursive applications](https://burr.dagworks.io/concepts/recursion).
+```
diff --git a/examples/recursive/application.py b/examples/recursive/application.py
new file mode 100644
index 00000000..1e53b9f7
--- /dev/null
+++ b/examples/recursive/application.py
@@ -0,0 +1,213 @@
+from typing import List, Tuple
+
+import joblib
+import openai
+from joblib import parallel_config
+
+from burr.core import Application, ApplicationBuilder, Condition, State, action
+from burr.core.application import ApplicationContext
+
+
+# full agent
+def _query_llm(prompt: str) -> str:
+ """Simple wrapper around the OpenAI API."""
+ client = openai.Client()
+ return (
+ client.chat.completions.create(
+ model="gpt-4o",
+ messages=[
+ {"role": "system", "content": "You are a helpful assistant"},
+ {"role": "user", "content": prompt},
+ ],
+ )
+ .choices[0]
+ .message.content
+ )
+
+
+@action(
+ reads=["feedback", "current_draft", "poem_type", "prompt"],
+ writes=["current_draft", "draft_history", "num_drafts"],
+)
+def write(state: State) -> Tuple[dict, State]:
+ """Writes a draft of a poem."""
+ poem_subject = state["prompt"]
+ poem_type = state["poem_type"]
+ current_draft = state.get("current_draft")
+ feedback = state.get("feedback")
+
+ parts = [
+ f'You are an AI poet. Create a {poem_type} poem on the following subject: "{poem_subject}".'
+ ]
+
+ if current_draft:
+ parts.append(f'Here is the current draft of the poem: "{current_draft}".')
+
+ if feedback:
+ parts.append(f'Please incorporate the following feedback: "{feedback}".')
+
+ parts.append(
+ f"Ensure the poem is creative, adheres to the style of a {poem_type}, and improves upon the previous draft."
+ )
+
+ prompt = "\n".join(parts)
+
+ draft = _query_llm(prompt)
+
+ return {"draft": draft}, state.update(
+ current_draft=draft,
+ draft_history=state.get("draft_history", []) + [draft],
+ ).increment(num_drafts=1)
+
+
+@action(reads=["current_draft", "poem_type", "prompt"], writes=["feedback"])
+def edit(state: State) -> Tuple[dict, State]:
+ """Edits a draft of a poem, providing feedback"""
+ poem_subject = state["prompt"]
+ poem_type = state["poem_type"]
+ current_draft = state["current_draft"]
+
+ prompt = f"""
+ You are an AI poetry critic. Review the following {poem_type} poem based on the subject: "{poem_subject}".
+ Here is the current draft of the poem: "{current_draft}".
+ Provide detailed feedback to improve the poem. If the poem is already excellent and needs no changes, simply respond with an empty string.
+ """
+
+ feedback = _query_llm(prompt)
+
+ return {"feedback": feedback}, state.update(feedback=feedback)
+
+
+@action(reads=["current_draft"], writes=["final_draft"])
+def final_draft(state: State) -> Tuple[dict, State]:
+ return {"final_draft": state["current_draft"]}, state.update(final_draft=state["current_draft"])
+
+
+def _create_sub_application(
+ max_num_drafts: int,
+ spawning_application_context: ApplicationContext,
+ poem_type: str,
+ prompt: str,
+) -> Application:
+ """Utility to create sub-application -- note"""
+ out = (
+ ApplicationBuilder()
+ .with_actions(
+ edit,
+ write,
+ final_draft,
+ )
+ .with_transitions(
+ ("write", "edit", Condition.expr(f"num_drafts < {max_num_drafts}")),
+ ("write", "final_draft"),
+ ("edit", "final_draft", Condition.expr("len(feedback) == 0")),
+ ("edit", "write"),
+ )
+ .with_tracker(spawning_application_context.tracker.copy()) # remember to do `copy()` here!
+ .with_spawning_parent(
+ spawning_application_context.app_id,
+ spawning_application_context.sequence_id,
+ spawning_application_context.partition_key,
+ )
+ .with_entrypoint("write")
+ .with_state(
+ current_draft=None,
+ poem_type=poem_type,
+ prompt=prompt,
+ feedback=None,
+ )
+ .build()
+ )
+ return out
+
+
+# full agent
+@action(
+ reads=[],
+ writes=[
+ "max_drafts",
+ "poem_types",
+ "poem_subject",
+ ],
+)
+def user_input(
+ state: State, max_drafts: int, poem_types: List[str], poem_subject: str
+) -> Tuple[dict, State]:
+ """Collects user input for the poem generation process."""
+ return {
+ "max_drafts": max_drafts,
+ "poem_types": poem_types,
+ "poem_subject": poem_subject,
+ }, state.update(max_drafts=max_drafts, poem_types=poem_types, poem_subject=poem_subject)
+
+
+@action(reads=["max_drafts", "poem_types", "poem_subject"], writes=["proposals"])
+def generate_all_poems(state: State, __context: ApplicationContext) -> Tuple[dict, State]:
+ # create one each
+ apps = [
+ _create_sub_application(state["max_drafts"], __context, poem_type, state["poem_subject"])
+ for poem_type in state["poem_types"]
+ ]
+ # run them all in parallel
+ with parallel_config(backend="threading", n_jobs=3):
+ all_results = joblib.Parallel()(
+ joblib.delayed(app.run)(halt_after=["final_draft"])
+ for app, poem_type in zip(apps, state["poem_types"])
+ )
+ proposals = []
+ for *_, substate in all_results:
+ proposals.append(substate["final_draft"])
+
+ return {"proposals": proposals}, state.update(proposals=proposals)
+
+
+@action(reads=["proposals", "prompts"], writes=["final_results"])
+def final_results(state: State) -> Tuple[dict, State]:
+ # joins them into a string
+ proposals = state["proposals"]
+ final_results = "\n\n".join(
+ [f"{poem_type}:\n{proposal}" for poem_type, proposal in zip(state["poem_types"], proposals)]
+ )
+ return {"final_results": final_results}, state.update(final_results=final_results)
+
+
+def application() -> Application:
+ return (
+ ApplicationBuilder()
+ .with_actions(
+ user_input,
+ generate_all_poems,
+ final_results,
+ )
+ .with_transitions(
+ ("user_input", "generate_all_poems"),
+ ("generate_all_poems", "final_results"),
+ )
+ .with_tracker(project="demo:parallelism_poem_generation")
+ .with_entrypoint("user_input")
+ .build()
+ )
+
+
+if __name__ == "__main__":
+ app = application()
+ app.visualize(output_file_path="statemachine", format="png")
+ _create_sub_application(
+ 2,
+ app.context,
+ "sonnet",
+ "state machines",
+ ).visualize(output_file_path="statemachine_sub", format="png")
+ app.run(
+ halt_after=["final_results"],
+ inputs={
+ "max_drafts": 2,
+ "poem_types": [
+ "sonnet",
+ "limerick",
+ "haiku",
+ "acrostic",
+ ],
+ "poem_subject": "state machines",
+ },
+ )
diff --git a/examples/recursive/requirements.txt b/examples/recursive/requirements.txt
new file mode 100644
index 00000000..cb387dcd
--- /dev/null
+++ b/examples/recursive/requirements.txt
@@ -0,0 +1,2 @@
+burr[learn]
+openai
diff --git a/examples/recursive/statemachine.png b/examples/recursive/statemachine.png
new file mode 100644
index 00000000..8adb0b82
Binary files /dev/null and b/examples/recursive/statemachine.png differ
diff --git a/examples/recursive/statemachine_sub.png b/examples/recursive/statemachine_sub.png
new file mode 100644
index 00000000..6328886e
Binary files /dev/null and b/examples/recursive/statemachine_sub.png differ