Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

new docs

Omar Khattab4232345e65ab
diff --git a/.gitignore b/.gitignore
index b2d6de306..9e10860cb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -18,3 +18,7 @@
npm-debug.log*
yarn-debug.log*
yarn-error.log*
+
+.env
+*.log
+*.ipynb
\ No newline at end of file
diff --git a/docs/building-blocks/3-language_models.md b/docs/building-blocks/1-language_models.md
similarity index 59%
rename from docs/building-blocks/3-language_models.md
rename to docs/building-blocks/1-language_models.md
index 992307d90..bd6f73ba8 100644
--- a/docs/building-blocks/3-language_models.md
+++ b/docs/building-blocks/1-language_models.md
@@ -1,53 +1,51 @@
---
-sidebar_position: 4
+sidebar_position: 2
---
# Language Models
+The most powerful features in DSPy revolve around algorithmically optimizing the prompts (or weights) of LMs, especially when you're building programs that use the LMs within a pipeline.
-## Remote LMs.
-
-These models are managed services. You just need to sign up and obtain an API key.
+Let's first make sure you can set up your language model. DSPy support clients for many remote and local LMs.
-1. `dspy.OpenAI` for GPT-3.5 and GPT-4.
+## Setting up the LM client.
-2. `dspy.Cohere`
+You can just call the constructor that connects to the LM. Then, use `dspy.configure` to declare this as the default LM.
-3. `dspy.Anyscale` for hosted Llama2 models.
+For example, to use OpenAI language models, you can do it as follows.
-### Local LMs.
+```python
+gpt3_turbo = dspy.OpenAI(model='gpt-3.5-turbo-1106', max_tokens=300)
+dspy.configure(lm=gpt3_turbo)
+```
-You need to host these models on your own GPU(s). Below, we include pointers for how to do that.
+ ['Hello! How can I assist you today?']
-1. `dspy.HFClientTGI`: for HuggingFace models through the Text Generation Inference (TGI) system. [Tutorial: How do I install and launch the TGI server?](/api/hosting_language_models_locally/TGI)
+## Directly calling the LM.
-2. `dspy.HFClientVLLM`: for HuggingFace models through vLLM. [Tutorial: How do I install and launch the vLLM server?](/api/hosting_language_models_locally/vLLM)
+You can simply call the LM with a string to give it a raw prompt, i.e. a string.
-3. `dspy.HFModel` (experimental)
+```python
+gpt3_turbo("hello! this is a raw prompt to GPT-3.5")
+```
-4. `dspy.Ollama` (experimental)
+This is almost never the recommended way to interact with LMs in DSPy, but it is allowed.
-5. `dspy.ChatModuleClient` (experimental): [How do I install and use MLC?](/api/hosting_language_models_locally/MLC)
+## Using the LM with DSPy signatures.
-If there are other clients you want added, let us know!
+You can also use the LM via DSPy [signatures] and [modules], which we discuss in more depth in the remaining guides.
-## Setting up the LM client.
-
-You can just call the constructor that connects to the LM. Then, use
-`dspy.configure` to declare this as the default LM.
-
-For example, for OpenAI, you can do it as follows.
+```python
+# Define a module (ChainOfThought) and assign it a signature (return an answer, given a question).
+qa = dspy.ChainOfThought('question -> answer')
-``` python
-gpt3_turbo = dspy.OpenAI(model='gpt-3.5-turbo-1106', max_tokens=300)
-gpt4_turbo = dspy.OpenAI(model='gpt-4-1106-preview', max_tokens=300)
+# Run with the default LM configured with `dspy.configure` above.
+response = qa(question="How many floors are in the castle David Gregory inherited?")
+print(response.answer)
+```
-# cohere = dspy.Cohere(...)
-# anyscale = dspy.Anyscale(...)
-# tgi_llama2 = dspy.HFClientTGI(model="meta-llama/Llama-2-7b-hf", port=8080, url="http://localhost")
+ The castle David Gregory inherited has 7 floors.
-dspy.configure(lm=gpt3_turbo)
-```
## Using multiple LMs at once.
@@ -57,19 +55,22 @@ Instead of changing the default LM, you can just change it inside a block of cod
**Tip:** Using `dspy.configure` and `dspy.context` is thread-safe!
-``` python
-qa = dspy.ChainOfThought('question -> answer')
-
+```python
+# Run with the default LM configured above, i.e. GPT-3.5
response = qa(question="How many floors are in the castle David Gregory inherited?")
-print(response.answer)
+print('GPT-3.5:', response.answer)
+
+gpt4_turbo = dspy.OpenAI(model='gpt-4-1106-preview', max_tokens=300)
+# Run with GPT-4 instead
with dspy.context(lm=gpt4_turbo):
response = qa(question="How many floors are in the castle David Gregory inherited?")
- print(response.answer)
+ print('GPT-4-turbo:', response.answer)
```
- The castle David Gregory inherited has 7 floors.
- The number of floors in the castle David Gregory inherited cannot be determined with the information provided.
+ GPT-3.5: The castle David Gregory inherited has 7 floors.
+ GPT-4-turbo: The number of floors in the castle David Gregory inherited cannot be determined with the information provided.
+
## Tips and Tricks.
@@ -80,7 +81,7 @@ will get new outputs.)
To generate 5 outputs, you can use `n=5` in the module constructor, or
pass `config=dict(n=5)` when invoking the module.
-``` python
+```python
qa = dspy.ChainOfThought('question -> answer', n=5)
response = qa(question="How many floors are in the castle David Gregory inherited?")
@@ -99,14 +100,53 @@ return the same value! That\'s by design.
To loop and generate one output at a time with the same input, bypass
the cache by making sure each request is (slightly) unique, as below.
-``` python
+```python
for idx in range(5):
response = qa(question="How many floors are in the castle David Gregory inherited?", config=dict(temperature=0.7+0.0001*idx))
- print(response.answer)
+ print(f'{idx+1}.', response.answer)
```
- The specific number of floors in David Gregory's inherited castle is not provided here, so further research would be needed to determine the answer.
- It is not possible to determine the exact number of floors in the castle David Gregory inherited without specific information about the castle's layout and history.
- The castle David Gregory inherited has 5 floors.
- We need more information to determine the number of floors in the castle David Gregory inherited.
- The castle David Gregory inherited has a total of 6 floors.
+ 1. The specific number of floors in David Gregory's inherited castle is not provided here, so further research would be needed to determine the answer.
+ 2. It is not possible to determine the exact number of floors in the castle David Gregory inherited without specific information about the castle's layout and history.
+ 3. The castle David Gregory inherited has 5 floors.
+ 4. We need more information to determine the number of floors in the castle David Gregory inherited.
+ 5. The castle David Gregory inherited has a total of 6 floors.
+
+
+## Remote LMs.
+
+These models are managed services. You just need to sign up and obtain an API key.
+
+1. `dspy.OpenAI` for GPT-3.5 and GPT-4.
+
+2. `dspy.Cohere`
+
+3. `dspy.Anyscale` for hosted Llama2 models.
+
+
+### Local LMs.
+
+You need to host these models on your own GPU(s). Below, we include pointers for how to do that.
+
+1. `dspy.HFClientTGI`: for HuggingFace models through the Text Generation Inference (TGI) system. [Tutorial: How do I install and launch the TGI server?](/api/hosting_language_models_locally/TGI)
+
+2. `dspy.HFClientVLLM`: for HuggingFace models through vLLM. [Tutorial: How do I install and launch the vLLM server?](/api/hosting_language_models_locally/vLLM)
+
+3. `dspy.HFModel` (experimental)
+
+4. `dspy.Ollama` (experimental)
+
+5. `dspy.ChatModuleClient` (experimental): [How do I install and use MLC?](/api/hosting_language_models_locally/MLC)
+
+If there are other clients you want added, let us know!
+
+
+<!-- TODO: Usage examples for these all.
+
+```python
+
+# cohere = dspy.Cohere(...)
+# anyscale = dspy.Anyscale(...)
+# tgi_llama2 = dspy.HFClientTGI(model="meta-llama/Llama-2-7b-hf", port=8080, url="http://localhost")
+
+``` -->
\ No newline at end of file
diff --git a/docs/building-blocks/1-signatures.md b/docs/building-blocks/2-signatures.md
similarity index 100%
rename from docs/building-blocks/1-signatures.md
rename to docs/building-blocks/2-signatures.md
diff --git a/docs/building-blocks/2-modules.md b/docs/building-blocks/3-modules.md
similarity index 86%
rename from docs/building-blocks/2-modules.md
rename to docs/building-blocks/3-modules.md
index e763b1279..7537c6bde 100644
--- a/docs/building-blocks/2-modules.md
+++ b/docs/building-blocks/3-modules.md
@@ -96,20 +96,20 @@ True
The others are very similar. They mainly change the internal behavior with which your signature is implemented!
-1. **`dspy.Predict`**:
+1. **`dspy.Predict`**: Basic predictor. Does not modify the signature. Handles the key forms of learning (i.e., storing the instructions and demonstrations and updates to the LM).
-2. **`dspy.ChainOfThought`**:
+2. **`dspy.ChainOfThought`**: Teaches the LM to think step-by-step before committing to the signature's response.
-3. **`dspy.ProgramOfThought`**:
+3. **`dspy.ProgramOfThought`**: Teaches the LM to output code, whose execution results will dictate the response.
-4. **`dspy.ReAct`**:
+4. **`dspy.ReAct`**: An agent that can use tools to implement the given signature.
-5. **`dspy.MultiChainComparison`**:
+5. **`dspy.MultiChainComparison`**: Can compare multiple outputs from `ChainOfThought` to produce a final prediction.
We also have some function-style modules:
-6. **`dspy.majority`**:
+6. **`dspy.majority`**: Can do basic voting to return the most popular response from a set of predictions.
More example soon!
diff --git a/docs/building-blocks/4-data.md b/docs/building-blocks/4-data.md
new file mode 100644
index 000000000..bfd161322
--- /dev/null
+++ b/docs/building-blocks/4-data.md
@@ -0,0 +1,88 @@
+---
+sidebar_position: 5
+---
+
+# Data
+
+DSPy is a machine learning framework, so working in it involves training sets, development sets, and test sets.
+
+For each example in your data, we distinguish typically between three types of values: the inputs, the intermediate labels, and the final label. You can use DSPy effectively without any intermediate or final labels, but you will need at least a few example inputs.
+
+## How much data do I need and how do I collect data for my task?
+
+Concretely, you can use DSPy optimizers usefully with as few as 10 example inputs, but having 50-100 examples (or even better, 300-500 examples) goes a long way.
+
+How can you get examples like these? If your task is extremely unusual, please invest in preparing ~10 examples by hand. Often times, depending on your metric below, you just need inputs and not labels, so it's not that hard.
+
+However, chances are that your task is not actually that unique. You can almost always find somewhat adjacent datasets on, say, HuggingFace datasets or other forms of data that you can leverage here.
+
+If there's data whose licenses are permissive enough, we suggest you use them. Otherwise, you can also start using/deploying/demoing your system and collect some initial data that way.
+
+## DSPy `Example` objects
+
+The core data type for data in DSPy is `Example`. You will use **Examples** to represent items in your training set and test set.
+
+DSPy **Examples** are similar to Python `dict`s but have a few useful utilities. Your DSPy modules will return values of the type `Prediction`, which is a special sub-class of `Example`.
+
+When you use DSPy, you will do a lot of evaluation and optimization runs. Your individual datapoints will be of type `Example`:
+
+```python
+qa_pair = dspy.Example(question="This is a question?", answer="This is an answer.")
+
+print(qa_pair)
+print(qa_pair.question)
+print(qa_pair.answer)
+```
+**Output:**
+```text
+Example({'question': 'This is a question?', 'answer': 'This is an answer.'}) (input_keys=None)
+This is a question?
+This is an answer.
+```
+
+Examples can have any field keys and any value types, though usually values are strings.
+
+```text
+object = Example(field1=value1, field2=value2, field3=value3, ...)
+```
+
+You can now express your training set for example as:
+
+```python
+trainset = [dspy.Example(report="LONG REPORT 1", summary="short summary 1"), ...]
+```
+
+
+### Specifying Input Keys
+
+In traditional ML, there are separated "inputs" and "labels".
+
+In DSPy, the `Example` objects have a `with_inputs()` method, which can mark specific fields as inputs. (The rest are just metadata or labels.)
+
+```python
+# Single Input.
+print(qa_pair.with_inputs("question"))
+
+# Multiple Inputs; be careful about marking your labels as inputs unless you mean it.
+print(qa_pair.with_inputs("question", "answer"))
+```
+
+Values can be accessed using the `.`(dot) operator. You can access the value of key `name` in defined object `Example(name="John Doe", job="sleep")` through `object.name`.
+
+To access or exclude certain keys, use `inputs()` and `labels()` methods to return new Example objects containing only input or non-input keys, respectively.
+
+```python
+article_summary = dspy.Example(article= "This is an article.", summary= "This is a summary.").with_inputs("article")
+
+input_key_only = article_summary.inputs()
+non_input_key_only = article_summary.labels()
+
+print("Example object with Input fields only:", input_key_only)
+print("Example object with Non-Input fields only:", non_input_key_only))
+```
+
+**Output**
+```
+Example object with Input fields only: Example({'article': 'This is an article.'}) (input_keys=None)
+Example object with Non-Input fields only: Example({'summary': 'This is a summary.'}) (input_keys=None)
+```
diff --git a/docs/building-blocks/5-metrics.md b/docs/building-blocks/5-metrics.md
new file mode 100644
index 000000000..fc974a900
--- /dev/null
+++ b/docs/building-blocks/5-metrics.md
@@ -0,0 +1,144 @@
+---
+sidebar_position: 5
+---
+
+# Metrics
+
+DSPy is a machine learning framework, so you must think about your **automatic metrics** for evaluation (to track your progress) and optimization (so DSPy can make your programs more effective).
+
+
+## What is a metric and how do I define a metric for my task?
+
+What makes outputs from your system good or bad? Invest in defining metrics and improving them over time incrementally. It's really hard to consistently improve what you aren't able to define.
+
+A metric is just a function that will take examples from your data and take the output of your system, and return a score that quantifies how good the output is.
+
+For simple tasks, this could be just "accuracy" or "exact match" or "F1 score". This may be the case for simple classification or short-form QA tasks.
+
+However, for most applications, your system will output long-form outputs. There, your metric should probably be a smaller DSPy program that checks multiple properties of the output (quite possibly using AI feedback from LMs).
+
+Getting this right on the first try is unlikely, but you should start with something simple and iterate.
+
+
+## Simple metrics
+
+A DSPy metric is just a function in Python that takes `example` (e.g., from your training or dev set) and the output `pred` from your DSPy program, and outputs a `float` (or `int` or `bool`) score.
+
+Your metric should also accept an optional third argument called `trace`. You can ignore this for a moment, but it will enable some powerful tricks if you want to use your metric for optimization.
+
+Here's a simple example of a metric that's comparing `example.answer` and `pred.answer`.
+
+```python
+def validate_answer(example, pred, trace=None):
+ return example.answer.lower() == pred.answer.lower()
+```
+
+Some people find these utilities (built-in) convenient:
+
+- `dspy.evaluate.metrics.answer_exact_match`
+- `dspy.evaluate.metrics.answer_passage_match`
+
+Your metrics could be more complex, e.g. check for multiple properties.
+
+```python
+def validate_context_and_answer(example, pred, trace=None):
+ # check the gold label and the predicted answer are the same
+ answer_match = example.answer.lower() == pred.answer.lower()
+
+ # check the predicted answer comes from one of the retrieved contexts
+ context_match = any((pred.answer.lower() in c) for c in pred.context)
+
+ return answer_match and context_match
+```
+
+Defining a good metric is an iterative process, so doing some initial evaluations and looking at your data and your outputs are key.
+
+
+## Evaluation
+
+Once you have a metric, you can run evaluations in a simple Python loop.
+
+```python
+scores = []
+for x in devset:
+ pred = program(**x.inputs())
+ score = metric(x, pred)
+ scores.append(score)
+```
+
+If you need some utilities, you can also use the built-in `Evaluate` utility. It can help with things like parallel evaluation (multiple threads) or showing you a sample of inputs/outputs and the metric scores.
+
+```python
+from dspy.evaluate import Evaluate
+
+# Set up the evaluator, which can be re-used in your code.
+evaluator = Evaluate(devset=YOUR_DEVSET, num_threads=1, display_progress=True, display_table=5)
+
+# Launch evaluation.
+evaluator(YOUR_PROGRAM, metric=YOUR_METRIC)
+```
+
+
+## Intermediate: Using AI feedback for your metric
+
+For most applications, your system will output long-form outputs, so your metric should check multiple dimensions of the output using AI feedback from LMs.
+
+This simple signature could come in handy.
+
+```python
+# Define the signature for automatic assessments.
+class Assess(dspy.Signature):
+ """Assess the quality of a tweet along the specified dimension."""
+
+ assessed_text = dspy.InputField()
+ assessment_question = dspy.InputField()
+ assessment_answer = dspy.OutputField(desc="Yes or No")
+```
+
+For example, below is a simple metric that uses GPT-4-turbo to check if a generated tweet (1) answers a given question correctly and (2) whether it's also engaging. We also check that (3) `len(tweet) <= 280` characters.
+
+```python
+gpt4T = dspy.OpenAI(model='gpt-4-1106-preview', max_tokens=1000, model_type='chat')
+
+def metric(gold, pred, trace=None):
+ question, answer, tweet = gold.question, gold.answer, pred.output
+
+ engaging = "Does the assessed text make for a self-contained, engaging tweet?"
+ correct = f"The text should answer `{question}` with `{answer}`. Does the assessed text contain this answer?"
+
+ with dspy.context(lm=gpt4T):
+ correct = dspy.Predict(Assess)(assessed_text=tweet, assessment_question=correct)
+ engaging = dspy.Predict(Assess)(assessed_text=tweet, assessment_question=engaging)
+
+ correct, engaging = [m.assessment_answer.lower() == 'yes' for m in [correct, engaging]]
+ score = (correct + engaging) if correct and (len(tweet) <= 280) else 0
+
+ if trace is not None: return score >= 2
+ return score / 2.0
+```
+
+When compiling, `trace is not None`, and we want to be strict about judging things, so we will only return `True` if `score >= 2`. Otherwise, we return a score out of 1.0 (i.e., `score / 2.0`).
+
+
+## Advanced: Using a DSPy program as your metric
+
+If your metric is itself a DSPy program, one of the most powerful ways to iterate is to compile (optimize) your metric itself. That's usually easy because the output of the metric is usually a simple value (e.g., a score out of 5) so the metric's metric is easy to define and optimize by collecting a few examples.
+
+
+
+### Advanced: Accessing the `trace`
+
+When your metric is used during evaluation runs, DSPy will not try to track the steps of your program.
+
+But during compiling (optimization), DSPy will trace your LM calls. The trace will contain inputs/outputs to each DSPy predictor and you can leverage that to validate intermediate steps for optimization.
+
+
+```python
+def validate_hops(example, pred, trace=None):
+ hops = [example.question] + [outputs.query for *_, outputs in trace if 'query' in outputs]
+
+ if max([len(h) for h in hops]) > 100: return False
+ if any(dspy.evaluate.answer_exact_match_str(hops[idx], hops[:idx], frac=0.8) for idx in range(2, len(hops))): return False
+
+ return True
+```
diff --git a/docs/building-blocks/6-optimizers.md b/docs/building-blocks/6-optimizers.md
new file mode 100644
index 000000000..2d41c71de
--- /dev/null
+++ b/docs/building-blocks/6-optimizers.md
@@ -0,0 +1,95 @@
+---
+sidebar_position: 6
+---
+
+# Optimizers (formerly Teleprompters)
+
+A **DSPy optimizer** is an algorithm that can tune the parameters of a DSPy program (i.e., the prompts and/or the LM weights) to maximize the metrics you specify, like accuracy.
+
+There are many built-in optimizers in DSPy, which apply vastly different strategies. A typical DSPy optimizer takes three things:
+
+- Your **DSPy program**. This may be a single module (e.g., `dspy.Predict`) or a complex multi-module program.
+
+- Your **metric**. This is a function that evaluates the output of your program, and assigns it a score (higher is better).
+
+- A few **training inputs**. This may be very small (i.e., only 5 or 10 examples) and incomplete (only inputs to your program, without any labels).
+
+If you happen to have a lot of data, DSPy can leverage that. But you can start small and get strong results.
+
+**Note:** Formerly called **DSPy Teleprompters**. We are making an official name update, which will be reflected throughout the library and documentation.
+
+
+## **What** does a DSPy Optimizer tune? **How** does it tune them?
+
+Traditional deep neural networks (DNNs) can be optimized with gradient descent, given a loss function and some training data.
+
+DSPy programs consist of multiple calls to LMs, stacked togther as [DSPy modules]. Each DSPy module has internal parameters of three kinds: (1) the LM weights, (2) the instructions, and (3) demonstrations of the input/output behavior.
+
+Given a metric, DSPy can optimize all of these three with multi-stage optimization algorithms. These can combine gradient descent (for LM weights) and discrete LM-driven optimization, i.e. for crafting/updating instructions and for creating/validating demonstrations. DSPy Demonstrations are like few-shot examples, but they're far more powerful. They can be created from scratch, given your program, and their creation and selection can be optimized in many effective ways.
+
+In many cases, we found that compiling leads to better prompts than humans write. Not because DSPy optimizers are more creative than humans, but simply because they can try more things, much more systematically, and tune the metrics directly.
+
+
+## What DSPy Optimizers are currently available?
+
+All of these can be accessed via `from dspy.teleprompt import *`.
+
+#### Automatic Few-Shot Learning
+
+1. **`LabeledFewShot`**: Simply constructs few-shot examples from provided labeled Q/A pairs.
+
+2. **`BootstrapFewShot`**: Uses your program to self-generate complete demonstrations for every stage of your program. Will simply use the generated demonstrations (if they pass the metric) without any further optimization. Advanced: Supports using a teacher program (a different DSPy program that has compatible structure) and a teacher LM, for harder tasks.
+
+3. **`BootstrapFewShotWithRandomSearch`**: Applies `BootstrapFewShot` several times with random search over generated demonstrations, and selects the best program.
+
+
+#### Automatic Instruction Optimization
+
+4. **`SignatureOptimizer`**: Generates and refines new instructions for each step, and optimizes them with coordinate ascent.
+
+5. **`BayesianSignatureOptimizer`**: Generates instructions and few-shot examples in each step. The instruction generation is data-aware and demonstration-aware. Uses Bayesian Optimization to effectively search over the space of generation instructions/demonstrations across your modules.
+
+
+#### Automatic Finetuning
+
+6. **`BootstrapFinetune`**: Distills a prompt-based DSPy program into weight updates (for smaller LMs). The output is a DSPy program that has the same steps, but where each step is conducted by a finetuned model instead of a prompted LM.
+
+
+#### Program Transformations
+
+7. **`KNNFewShot`**.
+
+8. **`Ensemble`**.
+
+
+## Which optimizer should I use?
+
+As a rule of thumb, if you don't know where to start, use `BootstrapFewShotWithRandomSearch`.
+
+Here's the general guidance on getting started:
+
+* If you have very little data, e.g. 10 examples of your task, use `BootstrapFewShot`.
+
+* If you have slightly more data, e.g. 50 examples of your task, use `BootstrapFewShotWithRandomSearch`.
+
+* If you have more data than that, e.g. 300 examples or more, use `BayesianSignatureOptimizer`.
+
+* If you have been able to use one of these with a large LM (e.g., 7B parameters or above) and need a very efficient program, compile that down to a small LM with `BootstrapFinetune`.
+
+
+## How do I use an optimizer?
+
+They all share this general interface, with some differences in the keyword arguments (hyperparameters).
+
+Let's see this with the most common one, `BootstrapFewShotWithRandomSearch`.
+
+```python
+from dspy.teleprompt import BootstrapFewShotWithRandomSearch
+
+# Set up the optimizer: we want to "bootstrap" (i.e., self-generate) 8-shot examples of your program's steps.
+# The optimizer will repeat this 10 times (plus some initial attempts) before selecting its best attempt on the devset.
+config = dict(max_bootstrapped_demos=3, max_labeled_demos=3, num_candidate_programs=10, num_threads=4)
+
+teleprompter = BootstrapFewShotWithRandomSearch(metric=YOUR_METRIC_HERE, **config)
+optimized_program = teleprompter.compile(YOUR_PROGRAM_HERE, trainset=YOUR_TRAINSET_HERE)
+```
diff --git a/docs/building-blocks/solving_your_task.md b/docs/building-blocks/solving_your_task.md
index 52310376a..14873f198 100644
--- a/docs/building-blocks/solving_your_task.md
+++ b/docs/building-blocks/solving_your_task.md
@@ -2,7 +2,7 @@
sidebar_position: 1
---
-# Using DSPy in 7 Steps
+# Using DSPy in 8 Steps
Using DSPy well for solving a new task is just doing good machine learning with LMs.
@@ -77,7 +77,7 @@ Now that you have some data and a metric, run evaluation on your pipeline before
Look at the outputs and the metric scores. This will probably allow you to spot any major issues, and it will define a baseline for your next step.
-## 6) Compile with a DSPy optimizer.
+## 7) Compile with a DSPy optimizer.
Given some data and a metric, we can now optimize the program you built.
@@ -93,7 +93,7 @@ Here's the general guidance on getting started:
* If you have been able to use one of these with a large LM (e.g., 7B parameters or above) and need a very efficient program, compile that down to a small LM with `BootstrapFinetune`.
-## 7) Iterate.
+## 8) Iterate.
At this point, you are either very happy with everything (we've seen quite a few people get it right on first try with DSPy) or, more likely, you've made a lot of progress but you don't like something about the final program or the metric.