Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Add difference b/w prompts

Herumb Shandilya0bc314156cdc
diff --git a/docs/docs/quick-start/minimal-example.mdx b/docs/docs/quick-start/minimal-example.mdx
index 7b3eff67f..46d472dad 100644
--- a/docs/docs/quick-start/minimal-example.mdx
+++ b/docs/docs/quick-start/minimal-example.mdx
@@ -22,7 +22,7 @@ import openai
from dspy.datasets import DataLoader
# Set up the LM.
-lm = dspy.LM(model='openai/gpt-4o-mini', max_tokens=8000)
+lm = dspy.LM(model='openai/gpt-4o-mini', max_tokens=1200)
dspy.configure(lm=lm)
# Load math questions from the MATH dataset using `DataLoader`
@@ -54,14 +54,76 @@ The `math_trainset` and `math_devset` datasets contain lists of `dspy.Examples`,
With our environment set up, let's define a custom program that utilizes the [`ChainOfThought`](/docs/deep-dive/modules/ChainOfThought) module to perform step-by-step reasoning to generate answers:
```python
-class CoT(dspy.Module):
- def __init__(self):
- super().__init__()
- self.prog = dspy.ChainOfThought("problem -> solution")
+cot = dspy.ChainOfThought("problem -> solution")
+```
+
+You can execute this module by passing `problem` as parameter to the call directly:
+
+```python
+cot(problem=math_devset[-1].problem)
+```
+
+**Output:**
+```
+Prediction(
+ reasoning='To factor the polynomial \\( x^{10} + x^5 + 1 \\), we can utilize the roots of unity. Specifically, we can let \\( \\omega \\) be a primitive cube root of unity, satisfying the equation \\( \\omega^3 = 1 \\) and \\( \\omega^2 + \\omega + 1 = 0 \\). By substituting \\( \\omega \\) into the polynomial, we find that \\( \\omega^{10} + \\omega^5 + 1 = 0 \\), indicating that \\( x^2 + x + 1 \\) is a factor of \\( x^{10} + x^5 + 1 \\).\n\nNext, we can express \\( x^{10} + x^5 + 1 \\) in a form that reveals its factors. We rewrite the polynomial by grouping terms and factoring out common factors. After some algebraic manipulation, we arrive at the factorization:\n\n\\[\nx^{10} + x^5 + 1 = (x^2 + x + 1)(x^8 - x^7 + x^5 - x^4 + x^3 - x + 1).\n\\]\n\nThis shows that the polynomial can be expressed as the product of two polynomials with integer coefficients.',
+ answer='The factorization of \\( x^{10} + x^5 + 1 \\) is \\( (x^2 + x + 1)(x^8 - x^7 + x^5 - x^4 + x^3 - x + 1) \\).'
+)
+```
+
+You can see the prompt along with completion for this call using `inspect_history` method:
+
+```python
+lm.inspect_history(n=1)
+```
- def forward(self, problem):
- return self.prog(problem=problem)
+**Output:**
```
+System message:
+
+Your input fields are:
+1. `problem` (str)
+
+Your output fields are:
+1. `reasoning` (str)
+2. `answer` (str)
+
+All interactions will be structured in the following way, with the appropriate values filled in.
+
+[[ ## problem ## ]]
+{problem}
+
+[[ ## reasoning ## ]]
+{reasoning}
+
+[[ ## answer ## ]]
+{answer}
+
+[[ ## completed ## ]]
+
+In adhering to this structure, your objective is:
+ Given the fields `problem`, produce the fields `answer`.
+
+
+User message:
+
+[[ ## problem ## ]]
+Write $x^{10} + x^5 + 1$ as the product of two polynomials with integer coefficients.
+
+Response:
+
+[[ ## reasoning ## ]]
+To factor the polynomial \( x^{10} + x^5 + 1 \), we can utilize the roots of unity. Specifically, we can let \( \omega \) be a primitive cube root of unity, satisfying the equation \( \omega^3 = 1 \) and \( \omega^2 + \omega + 1 = 0 \). By substituting \( \omega \) into the polynomial, we find that \( \omega^{10} + \omega^5 + 1 = 0 \), indicating that \( x^2 + x + 1 \) is a factor of \( x^{10} + x^5 + 1 \).
+
+...
+
+[[ ## answer ## ]]
+The factorization of \( x^{10} + x^5 + 1 \) is \( (x^2 + x + 1)(x^8 - x^7 + x^5 - x^4 + x^3 - x + 1) \).
+
+[[ ## completed ## ]]
+```
+
+In the prompt above there are 3 fields. The `System Message` contains the formatting information of the module, while the `User Message` contains the input question. The `Response` contains the output generated by the model.
## Compile and Evaluate the Model
@@ -91,7 +153,7 @@ This should take ~10-15 minutes to complete which you can speed but by increasin
print(f"Optimization Cost: {sum([lm.history[i]['cost'] for i in range(len(lm.history))])}")
```
-`lm.history` stores the history of your model's interactions, including the cost of each interaction. We can just sum the cost of these interaction and get the total cost of optimization, which is ~22 cents.
+`lm.history` stores the history of your model's interactions, including the cost of each interaction. We can just sum the cost of these interaction and get the total cost of optimization, which is ~26 cents.
:::note
Make sure that you clear the `lm.history` using `lm.history.clear()` before doing the optimization to get the accurate cost. This is especial
@@ -141,13 +203,63 @@ MATH a popular task that the models are directly optimized for resulting in a re
## Inspect the Model's History
-For a deeper understanding of the model's interactions, we can review the most recent generations through inspecting the model's history:
+For understanding the `compiled_cot` module's interactions, we can review the most recent generations through inspecting the model's history:
```python
-turbo.inspect_history(n=1)
+compiled_cot(problem=math_devset[-1].problem)
+dspy.inspect_history(n=1) # or lm.inspect_history(n=1)
+```
+
+**Output:**
+```
+System message:
+
+Your input fields are:
+1. `problem` (str)
+
+Your output fields are:
+1. `reasoning` (str)
+2. `answer` (str)
+
+All interactions will be structured in the following way, with the appropriate values filled in.
+
+[[ ## problem ## ]]
+{problem}
+
+[[ ## reasoning ## ]]
+{reasoning}
+
+[[ ## answer ## ]]
+{answer}
+
+[[ ## completed ## ]]
+
+In adhering to this structure, your objective is:
+ Analyze the given mathematical problem related to quadratic functions, and provide a detailed step-by-step reasoning process that leads to the final answer. Ensure that your explanation includes identifying key components like coefficients, roots, and the vertex of the parabola, and culminate with the final result based on your analysis.
+
+
+User message:
+
+[[ ## problem ## ]]
+Example({'problem': 'Write $x^{10} + x^5 + 1$ as the product of two polynomials with integer coefficients.', 'level': 'Level 5', 'type': 'Intermediate Algebra', 'solution': 'Let $\\omega$ satisfy $x^2 + x + 1 = 0,$ so $\\omega^2 + \\omega + 1 = 0.$ Then $(\\omega - 1)(\\omega^2 + \\omega + 1) = \\omega^3 - 1 = 0,$ so $\\omega^3 = 1.$ Also,\n\\begin{align*}\n\\omega^{10} + \\omega^5 + 1 &= \\omega^9 \\cdot \\omega + \\omega^3 \\cdot \\omega^2 + 1 \\\\\n&= \\omega + \\omega^2 + 1 \\\\\n&= 0.\n\\end{align*}Therefore, $x^2 + x + 1$ is a factor of $x^{10} + x^5 + 1.$\n\nTo bring out this factorization, we can write\n\\begin{align*}\nx^{10} + x^5 + 1 &= x^{10} - x + x^5 - x^2 + x^2 + x + 1 \\\\\n&= x(x^9 - 1) + x^2 (x^3 - 1) + x^2 + x + 1 \\\\\n&= x(x^3 - 1)(x^6 + x^3 + 1) + x^2 (x - 1)(x^2 + x + 1) + x^2 + x + 1 \\\\\n&= x(x - 1)(x^2 + x + 1)(x^6 + x^3 + 1) + x^2 (x - 1)(x^2 + x + 1) + x^2 + x + 1 \\\\\n&= \\boxed{(x^2 + x + 1)(x^8 - x^7 + x^5 - x^4 + x^3 - x + 1)}.\n\\end{align*}'}) (input_keys={'problem'})
+
+Respond with the corresponding output fields, starting with the field `reasoning`, then `answer`, and then ending with the marker for `completed`.
+
+
+Response:
+
+[[ ## reasoning ## ]]
+To factor the polynomial \( x^{10} + x^5 + 1 \), we can utilize the roots of unity. We start by letting \( \omega \) be a primitive cube root of unity, satisfying the equation \( \omega^3 = 1 \) and \( \omega^2 + \omega + 1 = 0 \).
+
+...
+
+[[ ## answer ## ]]
+The factorization of \( x^{10} + x^5 + 1 \) is \( (x^2 + x + 1)(x^8 - x^7 + x^5 - x^4 + x^3 - x + 1) \).
+
+[[ ## completed ## ]]
```
-And there you have it! You've successfully created a working example using the DSPy library.
+If you see the objective in the above prompt is now modified to an optimal objective from the optimizer. And there you have it! You've successfully created a working example using the DSPy library.
This example showcases how to set up your environment, define a custom module, compile a model, and rigorously evaluate its performance using the provided dataset and teleprompter configurations.