DSPy: The framework for programming—not prompting—language models
Python
Update docs
Omar Khattabcacaec501593
diff --git a/docs/docs/learn/index.md b/docs/docs/learn/index.mdindex 5b61f76de..5b37e12e6 100644--- a/docs/docs/learn/index.md+++ b/docs/docs/learn/index.md@@ -4,13 +4,12 @@ sidebar_position: 1# Learning DSPy: An Overview-As a framework, DSPy exposes a very small API that you can learn quickly. However, using it for novel tasks involves the more open-ended journey of composing its tools and design patterns to build _your_ modular AI systems quickly and to optimize them for _your_ objectives. This can be done along three different levels of system development, presented below.+DSPy exposes a very small API that you can learn quickly, but building new AI systems is a more open-ended journey of composing the tools and design patterns in DSPy to optimize for _your_ objectives. The three stages of building AI systems in DSPy are:+1) **DSPy Programming.** This is all about defining your task, its constraints, exploring a few examples, and using that to inform your initial pipeline design.-1) **Programming in DSPy.** This is the stage in which your concern is to define your task, its constraints, explore a few examples of it, and use that to inform your initial pipeline design.+2) **DSPy Evaluation.** Once your system works reasonably well, this is the stage where you collect an initial development set, define your DSPy metric, and use these to iterate on your system more systematically.-2) **Evaluation in DSPy.** Once you have a system that works reasonably well, you collect an initial small development set, define your DSPy metric, and use these to iterate on your system more systematically.+3) **DSPy Optimization.** Once you have a way to evaluate your system, you use DSPy optimizers to tune the prompts or weights in your program.-3) **Optimization in DSPy.** Once you have a system and a way to evaluate it, you can start to use DSPy optimizers to tune the prompts or weights in your program.--We typically suggest that you learn and apply DSPy in this order. For example, it's unproductive to launch optimization runs using a poorly-design program or a bad metric.\ No newline at end of file+We suggest learning and applying DSPy in this order. For example, it's unproductive to launch optimization runs using a poorly-design program or a bad metric.\ No newline at end of filediff --git a/docs/docs/learn/programming/language_models.md b/docs/docs/learn/programming/language_models.mdindex 0656e72f4..2766b6fc3 100644--- a/docs/docs/learn/programming/language_models.md+++ b/docs/docs/learn/programming/language_models.md@@ -6,8 +6,8 @@ sidebar_position: 2The first step in any DSPy code is to set up your language model. For example, you can configure OpenAI's GPT-4o-mini as your default LM as follows.-```python-# Authenticate via the `OPENAI_API_KEY` env variable.+```python linenums="1"+# Authenticate via `OPENAI_API_KEY` env: import os; os.environ['OPENAI_API_KEY'] = 'here'lm = dspy.LM('openai/gpt-4o-mini')dspy.configure(lm=lm)```@@ -108,7 +108,7 @@ lm(messages=[{"role": "user", "content": "Say this is a test!"}]) # => ['This iIdiomatic DSPy involves using _modules_, which we discuss in the next guide.-```python+```python linenums="1"# Define a module (ChainOfThought) and assign it a signature (return an answer, given a question).qa = dspy.ChainOfThought('question -> answer')@@ -129,7 +129,7 @@ You can change the default LM globally with `dspy.configure` or change it insideUsing `dspy.configure` and `dspy.context` is thread-safe!-```python+```python linenums="1"dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))response = qa(question="How many floors are in the castle David Gregory inherited?")print('GPT-4o-mini:', response.answer)@@ -148,7 +148,7 @@ GPT-3.5-turbo: The castle David Gregory inherited has 7 floors.For any LM, you can configure any of the following attributes at initialization or in each subsequent call.-```python+```python linenums="1"gpt_4o_mini = dspy.LM('openai/gpt-4o-mini', temperature=0.9, max_tokens=3000, stop=None, cache=False)```@@ -159,7 +159,7 @@ By default LMs in DSPy are cached. If you repeat the same call, you will get theEvery LM object maintains the history of its interactions, including inputs, outputs, token usage (and $$$ cost), and metadata.-```python+```python linenums="1"len(lm.history) # e.g., 3 calls to the LMlm.history[-1].keys() # access the last call to the LM, with all metadatadiff --git a/docs/docs/learn/programming/modules.md b/docs/docs/learn/programming/modules.mdindex dc75985b0..909ba7c2b 100644--- a/docs/docs/learn/programming/modules.md+++ b/docs/docs/learn/programming/modules.md@@ -23,7 +23,7 @@ To use a module, we first **declare** it by giving it a signature. Then we **calsentence = "it's a charming and often affecting journey." # example from the SST-2 dataset.# 1) Declare with a signature.-classify = dspy.Predict('sentence -> sentiment')+classify = dspy.Predict('sentence -> sentiment: bool')# 2) Call with input argument(s).response = classify(sentence=sentence)@@ -33,7 +33,7 @@ print(response.sentiment)```**Output:**```text-Positive+True```When we declare a module, we can pass configuration keys to it.@@ -54,7 +54,7 @@ response = classify(question=question)# 3) Access the outputs.response.completions.answer```-**Output:**+**Possible Output:**```text['One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.','Its ability to efficiently retrieve relevant information from large document collections.',@@ -71,9 +71,9 @@ Let's inspect the (first) reasoning and answer!print(f"Reasoning: {response.reasoning}")print(f"Answer: {response.answer}")```-**Output:**+**Possible Output:**```text-Rationale: produce the answer. We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.+Reasoning: We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.Answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.```diff --git a/docs/docs/learn/programming/overview.md b/docs/docs/learn/programming/overview.mdindex 86c97037a..c804ee615 100644--- a/docs/docs/learn/programming/overview.md+++ b/docs/docs/learn/programming/overview.md@@ -4,11 +4,11 @@ sidebar_position: 1# Programming in DSPy-DSPy is fundamentally a bet on _writing code instead of strings_: your control flow is often the most important part of your AI system. To design this, the first step is to **define your task**. What is the expected input/output behavior of your system? For example, are you trying to build a chatbot over your data or perhaps a code assistant? Or maybe a system for translation, for highlighting snippets from search results, or for generating reports with citations?+DSPy is a bet on _writing code instead of strings_: building the right control flow is crucial. Start by **defining your task**. What are the inputs to your system? What should your system produce as output? Is it a chatbot over your data or perhaps a code assistant? Or maybe a system for translation, for highlighting snippets from search results, or for generating reports with citations?-The next step is to **define your initial pipeline**. Can your DSPy program just be a simple chain-of-thought module? Do you need the LM to use retrieval or perhaps other tools, like a calculator or a calendar API? Is there a typical workflow for solving your problem in multiple well-scoped steps, or do you want more open-ended tool use with agents for your task? Think about this space but always start simple. Almost every task should probably start with just a single `dspy.ChainOfThought` module then add complexity incrementally based on observations.+Next, **define your initial pipeline**. Can your DSPy program just be a single module or do you need to break it down into a few steps? Do you need retrieval or other tools, like a calculator or a calendar API? Is there a typical workflow for solving your problem in multiple well-scoped steps, or do you want more open-ended tool use with agents for your task? Think about this space but always start simple, perhaps with just a single `dspy.ChainOfThought` module, then add complexity incrementally based on observations.-As you define your task and your pipeline, **craft and try a handful of examples** of the inputs to your program. Consider using a powerful LM at this point, or a couple of different LMs, just to understand what's possible. Record the interesting (both easy and hard) examples you try. This will be useful when you are doing evaluation and optimization later.+As you do this, **craft and try a handful of examples** of the inputs to your program. Consider using a powerful LM at this point, or a couple of different LMs, just to understand what's possible. Record interesting (both easy and hard) examples you try. This will be useful when you are doing evaluation and optimization later.??? "Beyond encouraging good design patterns, how does DSPy help here?"