Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Async tutorial (#8184)

Chen Qianf874ea98d946
diff --git a/docs/docs/tutorials/async/index.md b/docs/docs/tutorials/async/index.md
new file mode 100644
index 000000000..089036b68
--- /dev/null
+++ b/docs/docs/tutorials/async/index.md
@@ -0,0 +1,136 @@
+# Async DSPy Programming
+
+DSPy provides native support for asynchronous programming, allowing you to build more efficient and
+scalable applications. This guide will walk you through how to leverage async capabilities in DSPy,
+covering both built-in modules and custom implementations.
+
+## Why Use Async in DSPy?
+
+Asynchronous programming in DSPy offers several benefits:
+- Improved performance through concurrent operations
+- Better resource utilization
+- Reduced waiting time for I/O-bound operations
+- Enhanced scalability for handling multiple requests
+
+## When Should I use Sync or Async?
+
+Choosing between synchronous and asynchronous programming in DSPy depends on your specific use case.
+Here's a guide to help you make the right choice:
+
+Use Synchronous Programming When
+
+- You're exploring or prototyping new ideas
+- You're conducting research or experiments
+- You're building small to medium-sized applications
+- You need simpler, more straightforward code
+- You want easier debugging and error tracking
+
+Use Asynchronous Programming When:
+
+- You're building a high-throughput service (high QPS)
+- You're working with tools that only support async operations
+- You need to handle multiple concurrent requests efficiently
+- You're building a production service that requires high scalability
+
+### Important Considerations
+
+While async programming offers performance benefits, it comes with some trade-offs:
+
+- More complex error handling and debugging
+- Potential for subtle, hard-to-track bugs
+- More complex code structure
+- Different code between ipython (Colab, Jupyter lab, Databricks notebooks, ...) and normal python runtime.
+
+We recommend starting with synchronous programming for most development scenarios and switching to async
+only when you have a clear need for its benefits. This approach allows you to focus on the core logic of
+your application before dealing with the additional complexity of async programming.
+
+## Using Built-in Modules Asynchronously
+
+Most DSPy built-in modules support asynchronous operations through the `acall()` method. This method
+maintains the same interface as the synchronous `__call__` method but operates asynchronously.
+
+Here's a basic example using `dspy.Predict`:
+
+```python
+import dspy
+import asyncio
+import os
+
+os.environ["OPENAI_API_KEY"] = "your_api_key"
+
+dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
+predict = dspy.Predict("question->answer")
+
+async def main():
+ # Use acall() for async execution
+ output = await predict.acall(question="why did a chicken cross the kitchen?")
+ print(output)
+
+
+asyncio.run(main())
+```
+
+### Working with Async Tools
+
+DSPy's `Tool` class seamlessly integrates with async functions. When you provide an async
+function to `dspy.Tool`, you can execute it using `acall()`. This is particularly useful
+for I/O-bound operations or when working with external services.
+
+```python
+import asyncio
+import dspy
+import os
+
+os.environ["OPENAI_API_KEY"] = "your_api_key"
+
+async def foo(x):
+ # Simulate an async operation
+ await asyncio.sleep(0.1)
+ print(f"I get: {x}")
+
+# Create a tool from the async function
+tool = dspy.Tool(foo)
+
+async def main():
+ # Execute the tool asynchronously
+ await tool.acall(x=2)
+
+asyncio.run(main())
+```
+
+Note: When using `dspy.ReAct` with tools, calling `acall()` on the ReAct instance will automatically
+execute all tools asynchronously using their `acall()` methods.
+
+## Creating Custom Async DSPy Modules
+
+To create your own async DSPy module, implement the `aforward()` method instead of `forward()`. This method
+should contain your module's async logic. Here's an example of a custom module that chains two async operations:
+
+```python
+import dspy
+import asyncio
+import os
+
+os.environ["OPENAI_API_KEY"] = "your_api_key"
+dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
+
+class MyModule(dspy.Module):
+ def __init__(self):
+ self.predict1 = dspy.ChainOfThought("question->answer")
+ self.predict2 = dspy.ChainOfThought("answer->simplified_answer")
+
+ async def aforward(self, question, **kwargs):
+ # Execute predictions sequentially but asynchronously
+ answer = await self.predict1.acall(question=question)
+ return await self.predict2.acall(answer=answer)
+
+
+async def main():
+ mod = MyModule()
+ result = await mod.acall(question="Why did a chicken cross the kitchen?")
+ print(result)
+
+
+asyncio.run(main())
+```
diff --git a/docs/docs/tutorials/index.md b/docs/docs/tutorials/index.md
index 0088e91e8..9a63989f2 100644
--- a/docs/docs/tutorials/index.md
+++ b/docs/docs/tutorials/index.md
@@ -35,5 +35,6 @@ Welcome to DSPy tutorials! We've organized our tutorials into three main categor
- [Deployment](/tutorials/deployment/)
- [Debugging & Observability](/tutorials/observability/)
- [Streaming](/tutorials/streaming/)
+ - [Async](/tutorials/async/)
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index d9ce7837b..9860e52c9 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -52,6 +52,7 @@ nav:
- Deployment: tutorials/deployment/index.md
- Debugging & Observability: tutorials/observability/index.md
- Streaming: tutorials/streaming/index.md
+ - Async: tutorials/async/index.md
- DSPy in Production: production/index.md
- Community:
- Community Resources: community/community-resources.md