Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Update docs (#1938)

Omar Khattab7f4decac664b
diff --git a/docs/docs/index.md b/docs/docs/index.md
index 6bca0f1d8..7978a6c4f 100644
--- a/docs/docs/index.md
+++ b/docs/docs/index.md
@@ -140,7 +140,7 @@ DSPy shifts your focus from tinkering with prompt strings to **programming with
)
```
- === "Retrieval-Augmented Generation"
+ === "RAG"
```python linenums="1"
def search_wikipedia(query: str) -> list[str]:
@@ -236,6 +236,61 @@ DSPy shifts your focus from tinkering with prompt strings to **programming with
```text
5761.328
```
+
+ === "Multi-Stage Pipelines"
+
+ ```python linenums="1"
+ class Outline(dspy.Signature):
+ """Outline a thorough overview of a topic."""
+
+ topic: str = dspy.InputField()
+ title: str = dspy.OutputField()
+ sections: list[str] = dspy.OutputField()
+ section_subheadings: dict[str, list[str]] = dspy.OutputField(desc="mapping from section headings to subheadings")
+
+ class DraftSection(dspy.Signature):
+ """Draft a top-level section of an article."""
+
+ topic: str = dspy.InputField()
+ section_heading: str = dspy.InputField()
+ section_subheadings: list[str] = dspy.InputField()
+ content: str = dspy.OutputField(desc="markdown-formatted section")
+
+ class DraftArticle(dspy.Module):
+ def __init__(self):
+ self.build_outline = dspy.ChainOfThought(Outline)
+ self.draft_section = dspy.ChainOfThought(DraftSection)
+
+ def forward(self, topic):
+ outline = self.build_outline(topic=topic)
+ sections = []
+ for heading, subheadings in outline.section_subheadings.items():
+ section, subheadings = f"## {heading}", [f"### {subheading}" for subheading in subheadings]
+ section = self.draft_section(topic=outline.title, section_heading=section, section_subheadings=subheadings)
+ sections.append(section.content)
+ return dspy.Prediction(title=outline.title, sections=sections)
+
+ draft_article = DraftArticle()
+ article = draft_article(topic="World Cup 2002")
+ ```
+
+ **Possible Output:**
+
+ A 1500-word article on the topic, e.g.
+
+ ```text
+ ## Qualification Process
+
+ The qualification process for the 2002 FIFA World Cup involved a series of..... [shortened here for presentation].
+
+ ### UEFA Qualifiers
+
+ The UEFA qualifiers involved 50 teams competing for 13..... [shortened here for presentation].
+
+ .... [rest of the article]
+ ```
+
+ Note that DSPy makes it straightforward to optimize multi-stage modules like this. As long as you can evaluate the _final_ output of the system, every DSPy optimizer can tune all of the intermediate modules.
??? "Using DSPy in practice: from quick scripting to building sophisticated systems."
diff --git a/docs/docs/tutorials/index.md b/docs/docs/tutorials/index.md
index f4231a0f3..de7005837 100644
--- a/docs/docs/tutorials/index.md
+++ b/docs/docs/tutorials/index.md
@@ -4,6 +4,8 @@
* [Entity Extraction](/tutorials/entity_extraction/)
+* [Privacy-Conscious Delegation](/tutorials/papillon/)
+
* [Deployment](/tutorials/deployment/)
* [Debugging & Observability](/tutorials/observability/)
diff --git a/docs/docs/tutorials/papillon/index.md b/docs/docs/tutorials/papillon/index.md
new file mode 100644
index 000000000..ea57be9e5
--- /dev/null
+++ b/docs/docs/tutorials/papillon/index.md
@@ -0,0 +1,7 @@
+Please refer to [this tutorial from the PAPILLON authors](https://github.com/Columbia-NLP-Lab/PAPILLON/blob/main/papillon_tutorial.ipynb) using DSPy.
+
+This tutorial demonstrates a few aspects of using DSPy in a more advanced context:
+
+1. It builds a multi-stage `dspy.Module` that involves a small local LM using an external tool.
+2. It builds a multi-stage _judge_ in DSPy, and uses it as a metric for evaluation.
+3. It uses this judge for optimizing the `dspy.Module`, using a large model as a teacher for a small local LM.
\ No newline at end of file
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index e7c8fcd0d..4fa3f6eda 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -48,6 +48,7 @@ nav:
- Retrieval-Augmented Generation: tutorials/rag/index.ipynb
- Math Reasoning: tutorials/math/index.ipynb
- Entity Extraction: tutorials/entity_extraction/index.ipynb
+ - Privacy-Conscious Delegation: tutorials/papillon/index.md
- Deployment: tutorials/deployment/index.md
- Debugging & Observability: tutorials/observability/index.md
- Community: