DSPy: The framework for programming—not prompting—language models
Python
updated docs for CoTWithHint, MultiChainComparison, ReAct modules and Ensemble teleprompter
Arnav Singhvia0369356901c
diff --git a/README.md b/README.mdindex b3540c875..19a80747c 100644--- a/README.md+++ b/README.md@@ -272,8 +272,8 @@ We have work-in-progress module documentation at [this PR](https://github.com/st- [`dspy.ChainOfThought`](docs/modules.md#dspychainofthought)- `dspy.majority` (functional self-consistency)- `dspy.ProgramOfThought` [[see open PR]](https://github.com/stanfordnlp/dspy/pull/116)-- `dspy.ReAct` [see open PR]-- `dspy.MultiChainReasoning` [coming soon]+- [`dspy.ReAct`](docs/modules.md#dspyreact)+- [`dspy.MultiChainComparison`](docs/modules.md#dspymultichaincomparison)- `dspy.SelfCritique` [coming soon]- `dspy.SelfRevision` [coming soon]@@ -285,7 +285,7 @@ We have work-in-progress module documentation at [this PR](https://github.com/st- [`dspy.teleprompt.BootstrapFewShotWithRandomSearch`](docs/teleprompters.md#telepromptbootstrapfewshotwithrandomsearch)- `dspy.teleprompt.LabeledFinetune` [coming soon]- [`dspy.teleprompt.BootstrapFinetune`](docs/teleprompters.md#telepromptbootstrapfinetune)-- `dspy.teleprompt.Ensemble` [coming soon]+- [`dspy.teleprompt.Ensemble`](docs/teleprompters.md#telepromptensemble)- `dspy.teleprompt.kNN` [coming soon]</details>diff --git a/docs/modules.md b/docs/modules.mdindex 3fe302c09..2c68cb441 100644--- a/docs/modules.md+++ b/docs/modules.md@@ -9,6 +9,9 @@ This documentation provides an overview of the DSPy Modules.| Predict | [Predict Section](#dspypredict) || Retrieve | [Retrieve Section](#dspyretrieve) || ChainOfThought | [ChainOfThought Section](#dspychainofthought) |+| ChainOfThoughtWithHint | [ChainOfThoughtWithHint Section](#dspychainofthoughtwithhint) |+| MultiChainComparison | [MultiChainComparison Section](#dspymultichaincomparison) |+| ReAct | [ReAct Section](#dspyreact) |## dspy.Predict@@ -192,4 +195,237 @@ pred = generate_answer(question=question)print(f"Question: {question}")print(f"Predicted Answer: {pred.answer}")+```++## dspy.ChainOfThoughtWithHint++### Constructor++The constructor initializes the `ChainOfThoughtWithHint` class and sets up its attributes, inheriting from the `Predict` class. This class enhances the `ChainOfThought` class by offering an additional option to provide hints for reasoning. Two distinct signature templates are created internally depending on the presence of the hint.++```python+class ChainOfThoughtWithHint(Predict):+ def __init__(self, signature, rationale_type=None, activated=True, **config):+ super().__init__(signature, **config)++ self.activated = activated++ signature = self.signature+ *keys, last_key = signature.kwargs.keys()++ DEFAULT_HINT_TYPE = dsp.Type(prefix="Hint:", desc="${hint}")++ DEFAULT_RATIONALE_TYPE = dsp.Type(prefix="Reasoning: Let's think step by step in order to",+ desc="${produce the " + last_key + "}. We ...")++ rationale_type = rationale_type or DEFAULT_RATIONALE_TYPE++ extended_kwargs1 = {key: signature.kwargs[key] for key in keys}+ extended_kwargs1.update({'rationale': rationale_type, last_key: signature.kwargs[last_key]})++ extended_kwargs2 = {key: signature.kwargs[key] for key in keys}+ extended_kwargs2.update({'hint': DEFAULT_HINT_TYPE, 'rationale': rationale_type, last_key: signature.kwargs[last_key]})++ self.extended_signature1 = dsp.Template(signature.instructions, **extended_kwargs1)+ self.extended_signature2 = dsp.Template(signature.instructions, **extended_kwargs2)+```++**Parameters:**+- `signature` (_Any_): Signature of predictive model.+- `rationale_type` (_dsp.Type_, _optional_): Rationale type for reasoning steps. Defaults to `None`.+- `activated` (_bool_, _optional_): Flag for activated chain of thought processing. Defaults to `True`.+- `**config` (_dict_): Additional configuration parameters for model.++### Method++#### `forward(self, **kwargs)`++This method extends the parent `Predict` class's forward pass, updating the signature dynamically based on the presence of `hint` in the keyword arguments and the `activated` attribute.++**Parameters:**+- `**kwargs`: Keyword arguments required for prediction.++**Returns:**+- The result of the `forward` method in the parent `Predict` class.++### Examples++```python+#Define a simple signature for basic question answering+class BasicQA(dspy.Signature):+ """Answer questions with short factoid answers."""+ question = dspy.InputField()+ answer = dspy.OutputField(desc="often between 1 and 5 words")++#Pass signature to ChainOfThought module+generate_answer = dspy.ChainOfThoughtWithHint(BasicQA)++# Call the predictor on a particular input alongside a hint.+question='What is the color of the sky?'+hint = "It's what you often see during a sunny day."+pred = generate_answer(question=question, hint=hint)++print(f"Question: {question}")+print(f"Predicted Answer: {pred.answer}")+```+++## dspy.MultiChainComparison++### Constructor++The constructor initializes the `MultiChainComparison` class and sets up its attributes. It inherits from the `Predict` class and adds specific functionality for multiple chain comparisons.++The class incorporates multiple student attempt reasonings and concludes with the selected best reasoning path out of the available attempts.++```python+from .predict import Predict+from ..primitives.program import Module++import dsp++class MultiChainComparison(Module):+ def __init__(self, signature, M=3, temperature=0.7, **config):+ super().__init__()++ self.M = M+ signature = Predict(signature).signature+ *keys, last_key = signature.kwargs.keys()++ extended_kwargs = {key: signature.kwargs[key] for key in keys}++ for idx in range(M):+ candidate_type = dsp.Type(prefix=f"Student Attempt #{idx+1}:", desc="${reasoning attempt}")+ extended_kwargs.update({f'reasoning_attempt_{idx+1}': candidate_type})++ rationale_type = dsp.Type(prefix="Accurate Reasoning: Thank you everyone. Let's now holistically", desc="${corrected reasoning}")+ extended_kwargs.update({'rationale': rationale_type, last_key: signature.kwargs[last_key]})++ signature = dsp.Template(signature.instructions, **extended_kwargs)+ self.predict = Predict(signature, temperature=temperature, **config)+ self.last_key = last_key+```++**Parameters:**+- `signature` (_Any_): Signature of predictive model.+- `M` (_int_, _optional_): Number of student reasoning attempts. Defaults to `3`.+- `temperature` (_float_, _optional_): Temperature parameter for prediction. Defaults to `0.7`.+- `**config` (_dict_): Additional configuration parameters for model.++### Method++#### `forward(self, completions, **kwargs)`++This method aggregates all the student reasoning attempts and calls the predict method with extended signatures to get the best reasoning.++**Parameters:**+- `completions`: List of completion objects which include student reasoning attempts.+- `**kwargs`: Additional keyword arguments.++**Returns:**+- The result of the `predict` method for the best reasoning.++### Examples++```python+class BasicQA(dspy.Signature):+ """Answer questions with short factoid answers."""+ question = dspy.InputField()+ answer = dspy.OutputField(desc="often between 1 and 5 words")++# Example completions generated by a model for reference+completions = [+ dspy.Prediction(rationale="I recall that during clear days, the sky often appears this color.", answer="blue"),+ dspy.Prediction(rationale="Based on common knowledge, I believe the sky is typically seen as this color.", answer="green"),+ dspy.Prediction(rationale="From images and depictions in media, the sky is frequently represented with this hue.", answer="blue"),+]++# Pass signature to MultiChainComparison module+compare_answers = dspy.MultiChainComparison(BasicQA)++# Call the MultiChainComparison on the completions+question = 'What is the color of the sky?'+final_pred = compare_answers(completions, question=question)++print(f"Question: {question}")+print(f"Final Predicted Answer (after comparison): {final_pred.answer}")+print(f"Final Rationale: {final_pred.rationale}")+```++## dspy.ReAct++### Constructor++The constructor initializes the `ReAct` class and sets up its attributes. It is specifically designed to compose the interleaved steps of Thought, Action, and Observation.++Internally, the class follows a sequential process: Thoughts (or reasoning) lead to Actions (such as queries or activities). These Actions then result in Observations (like results or responses), which subsequently feedback into the next Thought. This cycle is maintained for a predefined number of iterations.++```python+import dsp+import dspy+from ..primitives.program import Module+from .predict import Predict++class ReAct(Module):+ def __init__(self, signature, max_iters=5, num_results=3, tools=None):+ ...+```++**Parameters:**+- `signature` (_Any_): Signature of the predictive model.+- `max_iters` (_int_, _optional_): Maximum number of iterations for the Thought-Action-Observation cycle. Defaults to `5`.+- `num_results` (_int_, _optional_): Number of results to retrieve in the action step. Defaults to `3`.+- `tools` (_List[dspy.Tool]_, _optional_): List of tools available for actions. If none is provided, a default `Retrieve` tool with `num_results` is used.++### Methods++#### `_generate_signature(self, iters)`++Generates a signature for the Thought-Action-Observation cycle based on the number of iterations.++**Parameters:**+- `iters` (_int_): Number of iterations.++**Returns:**+- A dictionary representation of the signature.++#### `act(self, output, hop)`++Processes an action and returns the observation or final answer.++**Parameters:**+- `output` (_dict_): Current output from the Thought.+- `hop` (_int_): Current iteration number.++**Returns:**+- A string representing the final answer or `None`.++#### `forward(self, **kwargs)`++Main method to execute the Thought-Action-Observation cycle for a given set of input fields.++**Parameters:**+- `**kwargs`: Keyword arguments corresponding to input fields.++**Returns:**+- A `dspy.Prediction` object containing the result of the ReAct process.++### Examples++```python+# Define a simple signature for basic question answering+class BasicQA(dspy.Signature):+ """Answer questions with short factoid answers."""+ question = dspy.InputField()+ answer = dspy.OutputField(desc="often between 1 and 5 words")++# Pass signature to ReAct module+react_module = dspy.ReAct(BasicQA)++# Call the ReAct module on a particular input+question = 'What is the color of the sky?'+result = react_module(question=question)++print(f"Question: {question}")+print(f"Final Predicted Answer (after ReAct process): {result.answer}")```\ No newline at end of filediff --git a/docs/teleprompters.md b/docs/teleprompters.mdindex 610922007..d60ea7f9d 100644--- a/docs/teleprompters.md+++ b/docs/teleprompters.md@@ -10,6 +10,7 @@ This documentation provides an overview of the DSPy Teleprompters.| --- | --- || LabeledFewShot | [LabeledFewShot Section](#telepromptlabeledfewshot) || BootstrapFewShot | [BootstrapFewShot Section](#telepromptbootstrapfewshot) |+| Ensemble | [Ensemble Section](#telepromptensemble) || BootstrapFewShotWithRandomSearch | [BootstrapFewShotWithRandomSearch Section](#telepromptbootstrapfewshotwithrandomsearch) || BootstrapFinetune | [BootstrapFinetune Section](#telepromptbootstrapfinetune) |@@ -128,6 +129,50 @@ teleprompter = BootstrapFewShot(teacher_settings=dict({'lm': teacher}))compiled_rag = teleprompter.compile(student=RAG(), trainset=trainset)```+## teleprompt.Ensemble++### Constructor++The constructor initializes the `Ensemble` class and sets up its attributes. This teleprompter is designed to create ensembled versions of multiple programs, reducing various outputs from different programs into a single output.++```python+class Ensemble(Teleprompter):+ def __init__(self, *, reduce_fn=None, size=None, deterministic=False):+```++**Parameters:**+- `reduce_fn` (_callable_, _optional_): Function used to reduce multiple outputs from different programs into a single output. A common choice is `dspy.majority`. Defaults to `None`.+- `size` (_int_, _optional_): Number of programs to randomly select for ensembling. If not specified, all programs will be used. Defaults to `None`.+- `deterministic` (_bool_, _optional_): Specifies whether ensemble should operate deterministically. Currently, setting this to `True` will raise an error as this feature is pending implementation. Defaults to `False`.++### Method++#### `compile(self, programs)`++This method compiles an ensemble of programs into a single program that when run, can either randomly sample a subset of the given programs to produce outputs or use all of them. The multiple outputs can then be reduced into a single output using the `reduce_fn`.++**Parameters:**+- `programs` (_list_): List of programs to be ensembled.++**Returns:**+- `EnsembledProgram` (_Module_): An ensembled version of the input programs.++### Example++```python+import dspy+from dspy.teleprompt import Ensemble++# Assume a list of programs+programs = [program1, program2, program3, ...]++# Define Ensemble teleprompter+teleprompter = Ensemble(reduce_fn=dspy.majority, size=2)++# Compile to get the EnsembledProgram+ensembled_program = teleprompter.compile(programs)+```+## teleprompt.BootstrapFewShotWithRandomSearch### Constructor