DSPy: The framework for programming—not prompting—language models
Python
Update the Learn DSPy docs (#1790)
Omar Khattab5ebf2f75b58a
diff --git a/docs/docs/building-blocks/1-language_models.md b/docs/docs/building-blocks/1-language_models.mddeleted file mode 100644index 604ece3cc..000000000--- a/docs/docs/building-blocks/1-language_models.md+++ /dev/null@@ -1,421 +0,0 @@-----sidebar_position: 2------# Language Models--The most powerful features in DSPy revolve around algorithmically optimizing the prompts (or weights) of LMs, especially when you're building programs that use the LMs within a pipeline.--Let's first make sure you can set up your language model. DSPy supports clients for many remote and local LMs.--## Using `dspy.LM`--!!! warning- Earlier versions of DSPy involved different clients across LM providers, e.g. `dspy.OpenAI`, `dspy.GoogleVertexAI`, and `dspy.HFClientTGI`, etc. These are now deprecated and will be removed in DSPy 2.6.-- Instead, use `dspy.LM` to access any LM endpoint for local and remote models. This relies on [LiteLLM](https://github.com/BerriAI/litellm) to translate the different client APIs into an OpenAI-compatible interface.-- Any [provider supported in LiteLLM](https://docs.litellm.ai/docs/providers) should work with `dspy.LM`.---### Setting up the LM client--In DSPy 2.5, we use the `dspy.LM` class to set up language models. This replaces the previous client-specific classes. Then, use `dspy.configure` to declare this as the default LM.--For example, to use OpenAI language models, you can do it as follows.--```python-lm = dspy.LM('openai/gpt-4o-mini')-dspy.configure(lm=lm)-```--### Directly calling the LM--You can simply call the LM with a string to give it a raw prompt, i.e. a string.--```python-lm("hello! this is a raw prompt to GPT-4o-mini")-```--**Output:**-```text-["Hello! It looks like you're trying to interact with a model. How can I assist you today?"]-```--For chat LMs, you can pass a list of messages.--```python-lm(messages=[{"role": "system", "content": "You are a helpful assistant."},- {"role": "user", "content": "What is 2+2?"}])-```--**Output:**-```text-['2 + 2 equals 4.']-```--This is almost never the recommended way to interact with LMs in DSPy, but it is allowed.---### Using the LM with DSPy signatures--You can also use the LM via DSPy [`signature` (input/output spec)](/building-blocks/2-signatures) and [`modules`](/building-blocks/3-modules), which we discuss in more depth in the remaining guides.--```python-# Define a module (ChainOfThought) and assign it a signature (return an answer, given a question).-qa = dspy.ChainOfThought('question -> answer')--# Run with the default LM configured with `dspy.configure` above.-response = qa(question="How many floors are in the castle David Gregory inherited?")-print(response.answer)-```-**Output:**-```text-The castle David Gregory inherited has 7 floors.-```--### Using multiple LMs at once.--The default LM above is GPT-3.5, `gpt3_turbo`. What if I want to run a piece of code with, say, GPT-4 or LLama-2?--Instead of changing the default LM, you can just change it inside a block of code.--!!! tip- Using `dspy.configure` and `dspy.context` is thread-safe!---```python-# Run with the default LM configured above, i.e. GPT-3.5-response = qa(question="How many floors are in the castle David Gregory inherited?")-print('GPT-3.5:', response.answer)--gpt4_turbo = dspy.LM(model='gpt-4-1106-preview', max_tokens=300)--# Run with GPT-4 instead-with dspy.context(lm=gpt4_turbo):- response = qa(question="How many floors are in the castle David Gregory inherited?")- print('GPT-4-turbo:', response.answer)-```-**Output:**-```text-GPT-3.5: The castle David Gregory inherited has 7 floors.-GPT-4-turbo: The number of floors in the castle David Gregory inherited cannot be determined with the information provided.-```--### Configuring LM attributes--For any LM, you can configure any of the following attributes at initialization or per call.--```python-gpt_4o_mini = dspy.LM('openai/gpt-4o-mini', temperature=0.9, max_tokens=3000, stop=None, cache=False)-```--By default LMs in DSPy are cached. If you repeat the same call, you will get the same outputs. But you can turn off caching by setting `cache=False` while declaring `dspy.LM` object.--### Using locally hosted LMs--Any OpenAI-compatible endpoint is easy to set up with an `openai/` prefix as well. This works great for open LMs from HuggingFace hosted locally with SGLang, VLLM, or HF Text-Generation-Inference.--#### Setting up SGLang--1. **Install SGLang (adapted from SGLang [documentation](https://sgl-project.github.io/start/install.html)):**-- ```bash- pip install "sglang[all]"- pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/- ```--2. **Login to HuggingFace:**-- ```bash- huggingface-cli login- ```-- You may need to set your HuggingFace API token to access gated models:-- ```bash- export HUGGINGFACEHUB_API_TOKEN=your_api_token- ```-- Also, ensure your cache directory is accessible:-- ```bash- export TRANSFORMERS_CACHE=/path/to/your/cache- ```--3. **Launch the SGLang server:**-- Replace `meta-llama/Meta-Llama-3-8B-Instruct` with the model you wish to launch.-- ```bash- CUDA_VISIBLE_DEVICES=0 python -m sglang.launch_server \- --model-path meta-llama/Meta-Llama-3-8B-Instruct \- --port 7501 \- ```--4. **Connect to the SGLang server via `dspy.LM`:**-- ```python- sglang_port = 7501- sglang_url = f"http://localhost:{sglang_port}/v1"-- # To avoid potential errors from LiteLLM when querying the SGLang endpoint, you may need to set api_key = ""-- sglang_llama = dspy.LM("openai/meta-llama/Meta-Llama-3-8B-Instruct", api_base=sglang_url, api_key = "")-- # You could also use text mode, in which the prompts are *not* formatted as messages.- sglang_llama_text = dspy.LM("openai/meta-llama/Meta-Llama-3-8B-Instruct", api_base=sglang_url, model_type='text', api_key = "")- ```-- For further details on customizing the SGLang configuration, please refer to the [SGLang documentation](https://sgl-project.github.io/backend/backend.html#additional-server-arguments).--#-Additionally, you can also host models locally through Ollama.--#### Setting Up Ollama (adapted from Ollama [documentation](https://github.com/ollama/ollama))---1. **Install Ollama:**--- ```bash- curl -fsSL https://ollama.ai/install.sh | sh- ```--2. **Launch the Ollama server:**-- Replace `llama3.2:1b` with the model you wish to launch.-- ```bash- ollama run llama3.2:1b- ```-- If the model isn't in GGUF format, you'll need to convert it first, following corresponding Ollama [instructions](https://github.com/ollama/ollama?tab=readme-ov-file#import-from-gguf).-- If you wish to download the model manually and load it into Ollama, please follow the corresponding Ollama import [instructions](https://github.com/ollama/ollama/blob/main/docs/import.md).--3. **Connect to the Ollama server via `dspy.LM`:**-- ```python- ollama_port = 11434- ollama_url = f"http://localhost:{ollama_port}"- ollama_llm = dspy.LM(model="ollama/llama3.2:1b", api_base=ollama_url)- ```-- For further details on customizing the Ollama configuration, please refer to the LiteLLM [Ollama documentation](https://docs.litellm.ai/docs/providers/ollama).--### Inspecting output and usage metadata--Every LM object maintains the history of its interactions, including inputs, outputs, token usage (and $$$ cost), and metadata.--```python-len(lm.history) # e.g., 3 calls to the LM--lm.history[-1].keys() # access the last call to the LM, with all metadata-```--**Output:**-```text-dict_keys(['prompt', 'messages', 'kwargs', 'response', 'outputs', 'usage', 'cost'])-```--## Creating Custom LM Class--Creating a custom LM class is quite straightforward in DSPy. You can inherit from the `dspy.LM` class or create a new class with a similar interface. You'll need to implement/override these three methods:--* `__init__`: Initialize the LM with the given `model` and other keyword arguments.-* `__call__`: Call the LM with the given input prompt and return a list of string outputs.-* `inspect_history`: The history of interactions with the LM. This is optional but is needed by some optimizers in DSPy.--!!! tip- If there is not much overlap in features between your LM and LiteLLM it's better to not inherit and implement all methods from ground up.---Let's create an LM for Gemini using `google-generativeai` package from scratch:--```python-import os-import dspy-import google.generativeai as genai--class GeminiLM(dspy.LM):- def __init__(self, model, api_key=None, endpoint=None, **kwargs):- genai.configure(api_key=os.environ["GEMINI_API_KEY"] or api_key)-- self.endpoint = endpoint- self.history = []-- super().__init__(model, **kwargs)- self.model = genai.GenerativeModel(model)-- def __call__(self, prompt=None, messages=None, **kwargs):- # Custom chat model working for text completion model- prompt = '\n\n'.join([x['content'] for x in messages] + ['BEGIN RESPONSE:'])-- completions = self.model.generate_content(prompt)- self.history.append({"prompt": prompt, "completions": completions})-- # Must return a list of strings- return [completions.candidates[0].content.parts[0].text]-- def inspect_history(self):- for interaction in self.history:- print(f"Prompt: {interaction['prompt']} -> Completions: {interaction['completions']}")--lm = GeminiLM("gemini-1.5-flash", temperature=0)-dspy.configure(lm=lm)--qa = dspy.ChainOfThought("question->answer")-qa(question="What is the capital of France?")-```--**Output:**-```text-Prediction(- reasoning='France is a country in Western Europe. Its capital city is Paris.',- answer='Paris'-)-```--The above example is the simplest form of LM. You can add more options to tweak generation config and even control the generated output based on your requirement.---## Structured LM output with Adapters--Prompt optimizers in DSPy generate and tune the _instructions_ or the _examples_ in the prompts corresponding to your Signatures. DSPy 2.5 introduces **Adapters** as a layer between Signatures and LMs, responsible for formatting these pieces (Signature I/O fields, instructions, and examples) as well as generating and parsing the outputs.--In DSPy 2.5, the default Adapters are now more natively aware of chat LMs and are responsible for enforcing types, building on earlier experimental features from `TypedPredictor` and `configure(experimental=True)`. In our experience, this change tends to deliver more consistent pre-optimization quality from various LMs, especially for sophisticated Signatures.--```python-lm = dspy.LM('openai/gpt-4o-mini')-dspy.configure(lm=lm, experimental=True)--fact_checking = dspy.ChainOfThought('claims -> verdicts')-fact_checking(claims=["Python was released in 1991.", "Python is a compiled language."])-```--**Output:**-```text-Prediction(- reasoning='The first claim states that "Python was released in 1991," which is true. Python was indeed first released by Guido van Rossum in February 1991. The second claim states that "Python is a compiled language." This is false; Python is primarily an interpreted language, although it can be compiled to bytecode, it is not considered a compiled language in the traditional sense like C or Java.',- verdicts=[True, False]-)-```--### Defining Custom Adapters--!!! warning- Adapters are low-level features that change the way input and output is handled by DSPy, it's not recommended to build and use custom Adapters unless you are sure of what you are doing.---Adapters are a powerful feature in DSPy, allowing you to define custom behavior for your Signatures.--For example, you could define an Adapter that automatically converts the input to uppercase before passing it to the LM. This is a simple example, but it shows how you can create custom Adapters that modify the inputs or outputs of your LMs.--You'll need to inherit the base `Adapter` class and implement two methods to create a usable custom Adapter:--* `format`: This method is responsible for formatting the input for the LM. This method takes `signature`, `demos` and `inputs` as input parameters. Demos are in-context examples set manually or through example. The output of this function can be a string prompt supported by completions function, a list of message dictionaries or any format that the LM you are using supports.--* `parse`: This method is responsible for parsing the output of the LM. This method takes `signature`, `completions` and `_parse_values` as input parameters.--```python-from dspy.adapters.base import Adapter-from typing import List, Dict--class UpperCaseAdapter(Adapter):- def __init__(self):- super().__init__()-- def format(self, signature, demos, inputs):- system_prompt = signature.instructions- all_fields = signature.model_fields- all_field_data = [(all_fields[f].json_schema_extra["prefix"], all_fields[f].json_schema_extra["desc"]) for f in all_fields]-- all_field_data_str = "\n".join([f"{p}: {d}" for p, d in all_field_data])- format_instruction_prompt = "="*20 + f"""\n\nOutput Format:\n\n{all_field_data_str}\n\n""" + "="*20-- all_input_fields = signature.input_fields- input_fields_data = [(all_input_fields[f].json_schema_extra["prefix"], inputs[f]) for f in all_input_fields]-- input_fields_str = "\n".join([f"{p}: {v}" for p, v in input_fields_data])-- # Convert to uppercase- return (system_prompt + format_instruction_prompt + input_fields_str).upper()-- def parse(self, signature, completions, _parse_values=None):- output_fields = signature.output_fields-- output_dict = {}- for field in output_fields:- field_info = output_fields[field]- prefix = field_info.json_schema_extra["prefix"]-- field_completion = completions.split(prefix.upper())[-1].split("\n")[0].strip(": ")- output_dict[field] = field_completion-- return output_dict-```--Let's understand the `UpperCaseAdapter` class. The `format` method takes `signature`, `demos`, and `inputs` as input parameters. It constructs a prompt by combining the system prompt, format instruction prompt, and input fields, and then converts the prompt to uppercase.--The `parse` method takes `signature`, `completions`, and `_parse_values` as input parameters. It then extracts the output fields from the completions and returns them as a dictionary.--Once you have defined your custom Adapter, you can use it in your Signatures by passing it as an argument to the `dspy.configure` method.--```python-dspy.configure(adapter=UpperCaseAdapter())-```--Now, when you run an inference over a Signature, the input will be converted to uppercase before being passed to the LM. The output will be parsed as a dictionary.--```python-lm = dspy.LM('openai/gpt-4o-mini')-dspy.configure(lm=lm, adapter=UpperCaseAdapter())--qa = dspy.ChainOfThought('question -> answer')--response = qa(question="How many floors are in the castle David Gregory inherited?")-response-```--**Output:**-```text-Prediction(- reasoning='determine the number of floors in the castle that David Gregory inherited. This information typically comes from a specific source or context, such as a book, movie, or historical reference. Without that context, I cannot provide an exact number.',- answer='I do not have the specific information regarding the number of floors in the castle David Gregory inherited.'-)-```--Now let's see how the prompt after Adapter looks like!--```python-lm.inspect_history()-```--**Output:**-```text-User message:--GIVEN THE FIELDS `QUESTION`, PRODUCE THE FIELDS `ANSWER`.====================--OUTPUT FORMAT:--QUESTION:: ${QUESTION}-REASONING: LET'S THINK STEP BY STEP IN ORDER TO: ${REASONING}-ANSWER:: ${ANSWER}--====================QUESTION:: HOW MANY FLOORS ARE IN THE CASTLE DAVID GREGORY INHERITED?---Response:--QUESTION:: HOW MANY FLOORS ARE IN THE CASTLE DAVID GREGORY INHERITED?-REASONING: LET'S THINK STEP BY STEP IN ORDER TO: determine the number of floors in the castle that David Gregory inherited. This information typically comes from a specific source or context, such as a book, movie, or historical reference. Without that context, I cannot provide an exact number.-ANSWER:: I do not have the specific information regarding the number of floors in the castle David Gregory inherited.-```--The above example is a simple Adapter that converts the input to uppercase before passing it to the LM. You can define more complex Adapters based on your requirements.--### Overriding `__call__` method--To gain control over usage of format and parse and even more fine-grained control over the flow of input from signature to outputs you can override `__call__` method and implement your custom flow. Although for most cases only implementing `parse` and `format` function will be fine.diff --git a/docs/docs/building-blocks/3-modules.md b/docs/docs/building-blocks/3-modules.mddeleted file mode 100644index c3cc5975f..000000000--- a/docs/docs/building-blocks/3-modules.md+++ /dev/null@@ -1,126 +0,0 @@-----sidebar_position: 3------# Modules--A **DSPy module** is a building block for programs that use LMs.--- Each built-in module abstracts a **prompting technique** (like chain of thought or ReAct). Crucially, they are generalized to handle any [DSPy Signature](/building-blocks/2-signatures).--- A DSPy module has **learnable parameters** (i.e., the little pieces comprising the prompt and the LM weights) and can be invoked (called) to process inputs and return outputs.--- Multiple modules can be composed into bigger modules (programs). DSPy modules are inspired directly by NN modules in PyTorch, but applied to LM programs.---## How do I use a built-in module, like `dspy.Predict` or `dspy.ChainOfThought`?--Let's start with the most fundamental module, `dspy.Predict`. Internally, all other DSPy modules are just built using `dspy.Predict`.--We'll assume you are already at least a little familiar with [DSPy signatures](/building-blocks/2-signatures), which are declarative specs for defining the behavior of any module we use in DSPy.--To use a module, we first **declare** it by giving it a signature. Then we **call** the module with the input arguments, and extract the output fields!--```python-sentence = "it's a charming and often affecting journey." # example from the SST-2 dataset.--# 1) Declare with a signature.-classify = dspy.Predict('sentence -> sentiment')--# 2) Call with input argument(s).-response = classify(sentence=sentence)--# 3) Access the output.-print(response.sentiment)-```-**Output:**-```text-Positive-```--When we declare a module, we can pass configuration keys to it.--Below, we'll pass `n=5` to request five completions. We can also pass `temperature` or `max_len`, etc.--Let's use `dspy.ChainOfThought`. In many cases, simply swapping `dspy.ChainOfThought` in place of `dspy.Predict` improves quality.--```python-question = "What's something great about the ColBERT retrieval model?"--# 1) Declare with a signature, and pass some config.-classify = dspy.ChainOfThought('question -> answer', n=5)--# 2) Call with input argument.-response = classify(question=question)--# 3) Access the outputs.-response.completions.answer-```-**Output:**-```text-['One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.',- 'Its ability to efficiently retrieve relevant information from large document collections.',- 'One great thing about the ColBERT retrieval model is its superior performance compared to other models and its efficient use of pre-trained language models.',- 'One great thing about the ColBERT retrieval model is its superior efficiency and accuracy compared to other models.',- 'One great thing about the ColBERT retrieval model is its ability to incorporate user feedback and support complex queries.']-```--Let's discuss the output object here.--The `dspy.ChainOfThought` module will generally inject a `reasoning` before the output field(s) of your signature.--Let's inspect the (first) reasoning and answer!--```python-print(f"Reasoning: {response.reasoning}")-print(f"Answer: {response.answer}")-```-**Output:**-```text-Rationale: produce the answer. We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.-Answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.-```--This is accessible whether we request one or many completions.--We can also access the different completions as a list of `Prediction`s or as several lists, one for each field.--```python-response.completions[3].reasoning == response.completions.reasoning[3]-```-**Output:**-```text-True-```---## What other DSPy modules are there? How can I use them?--The others are very similar. They mainly change the internal behavior with which your signature is implemented!--1. **`dspy.Predict`**: Basic predictor. Does not modify the signature. Handles the key forms of learning (i.e., storing the instructions and demonstrations and updates to the LM).--2. **`dspy.ChainOfThought`**: Teaches the LM to think step-by-step before committing to the signature's response.--3. **`dspy.ProgramOfThought`**: Teaches the LM to output code, whose execution results will dictate the response.--4. **`dspy.ReAct`**: An agent that can use tools to implement the given signature.--5. **`dspy.MultiChainComparison`**: Can compare multiple outputs from `ChainOfThought` to produce a final prediction.---We also have some function-style modules:--6. **`dspy.majority`**: Can do basic voting to return the most popular response from a set of predictions.---Check out further examples in [each module's respective guide](/deep-dive/modules/guide).---## How do I compose multiple modules into a bigger program?--DSPy is just Python code that uses modules in any control flow you like. (There's some magic internally at `compile` time to trace your LM calls.)--What this means is that, you can just call the modules freely. No weird abstractions for chaining calls.--This is basically PyTorch's design approach for define-by-run / dynamic computation graphs. Refer to the intro tutorials for examples.diff --git a/docs/docs/building-blocks/6-optimizers.md b/docs/docs/building-blocks/6-optimizers.mddeleted file mode 100644index 14f0036b6..000000000--- a/docs/docs/building-blocks/6-optimizers.md+++ /dev/null@@ -1,128 +0,0 @@-----sidebar_position: 6------# Optimizers (formerly Teleprompters)--A **DSPy optimizer** is an algorithm that can tune the parameters of a DSPy program (i.e., the prompts and/or the LM weights) to maximize the metrics you specify, like accuracy.--There are many built-in optimizers in DSPy, which apply vastly different strategies. A typical DSPy optimizer takes three things:--- Your **DSPy program**. This may be a single module (e.g., `dspy.Predict`) or a complex multi-module program.--- Your **metric**. This is a function that evaluates the output of your program, and assigns it a score (higher is better).--- A few **training inputs**. This may be very small (i.e., only 5 or 10 examples) and incomplete (only inputs to your program, without any labels).--If you happen to have a lot of data, DSPy can leverage that. But you can start small and get strong results.--**Note:** Formerly called **DSPy Teleprompters**. We are making an official name update, which will be reflected throughout the library and documentation.---## **What** does a DSPy Optimizer tune? **How** does it tune them?--Traditional deep neural networks (DNNs) can be optimized with gradient descent, given a loss function and some training data.--DSPy programs consist of multiple calls to LMs, stacked together as [DSPy modules]. Each DSPy module has internal parameters of three kinds: (1) the LM weights, (2) the instructions, and (3) demonstrations of the input/output behavior.--Given a metric, DSPy can optimize all of these three with multi-stage optimization algorithms. These can combine gradient descent (for LM weights) and discrete LM-driven optimization, i.e. for crafting/updating instructions and for creating/validating demonstrations. DSPy Demonstrations are like few-shot examples, but they're far more powerful. They can be created from scratch, given your program, and their creation and selection can be optimized in many effective ways.--In many cases, we found that compiling leads to better prompts than human writing. Not because DSPy optimizers are more creative than humans, but simply because they can try more things, much more systematically, and tune the metrics directly.---## What DSPy Optimizers are currently available?--<!-- The following diagram was generated by: -->-<!-- 1. Running symilar on the teleprompter module to extract the python hierarchy as a Graphviz dot file -->-<!-- 2. Hand-editing the resulting dot file to remove classes that are not teleprompters/optimizers (e.g., classes for data structures manipulated by optimizers). -->-<!-- 3. Using dot to compile the `.dot` file into a PNG -->-<!-- Robert Goldman [2024/05/11:rpg] -->----All of these can be accessed via `from dspy.teleprompt import *`.--### Automatic Few-Shot Learning--These optimizers extend the signature by automatically generating and including **optimized** examples within the prompt sent to the model, implementing few-shot learning.--1. **`LabeledFewShot`**: Simply constructs few-shot examples (demos) from provided labeled input and output data points. Requires `k` (number of examples for the prompt) and `trainset` to randomly select `k` examples from.--2. **`BootstrapFewShot`**: Uses a `teacher` module (which defaults to your program) to generate complete demonstrations for every stage of your program, along with labeled examples in `trainset`. Parameters include `max_labeled_demos` (the number of demonstrations randomly selected from the `trainset`) and `max_bootstrapped_demos` (the number of additional examples generated by the `teacher`). The bootstrapping process employs the metric to validate demonstrations, including only those that pass the metric in the "compiled" prompt. Advanced: Supports using a `teacher` program that is a *different* DSPy program that has compatible structure, for harder tasks.--3. [**`BootstrapFewShotWithRandomSearch`**](/deep-dive/optimizers/bootstrap-fewshot): Applies `BootstrapFewShot` several times with random search over generated demonstrations, and selects the best program over the optimization. Parameters mirror those of `BootstrapFewShot`, with the addition of `num_candidate_programs`, which specifies the number of random programs evaluated over the optimization, including candidates of the uncompiled program, `LabeledFewShot` optimized program, `BootstrapFewShot` compiled program with unshuffled examples and `num_candidate_programs` of `BootstrapFewShot` compiled programs with randomized example sets.--4. **`KNNFewShot`**. Uses k-Nearest Neighbors algorithm to find the nearest training example demonstrations for a given input example. These nearest neighbor demonstrations are then used as the trainset for the BootstrapFewShot optimization process. See [this notebook](https://github.com/stanfordnlp/dspy/blob/main/examples/knn.ipynb) for an example.---### Automatic Instruction Optimization--These optimizers produce optimal instructions for the prompt and, in the case of MIPROv2 can also optimize the set of few-shot demonstrations.--5. [**`COPRO`**](/deep-dive/optimizers/copro): Generates and refines new instructions for each step, and optimizes them with coordinate ascent (hill-climbing using the metric function and the `trainset`). Parameters include `depth` which is the number of iterations of prompt improvement the optimizer runs over.--6. [**`MIPROv2`**](/deep-dive/optimizers/miprov2): Generates instructions *and* few-shot examples in each step. The instruction generation is data-aware and demonstration-aware. Uses Bayesian Optimization to effectively search over the space of generation instructions/demonstrations across your modules.---### Automatic Finetuning--This optimizer is used to fine-tune the underlying LLM(s).--7. **`BootstrapFinetune`**: Distills a prompt-based DSPy program into weight updates (for smaller LMs). The output is a DSPy program that has the same steps, but where each step is conducted by a finetuned model instead of a prompted LM.---### Program Transformations--8. **`Ensemble`**: Ensembles a set of DSPy programs and either uses the full set or randomly samples a subset into a single program.---## Which optimizer should I use?--Ultimately, finding the ‘right’ optimizer to use & the best configuration for your task will require experimentation. Success in DSPy is still an iterative process - getting the best performance on your task will require you to explore and iterate.--That being said, here's the general guidance on getting started:-* If you have **very few examples** (around 10), start with `BootstrapFewShot`.-* If you have **more data** (50 examples or more), try `BootstrapFewShotWithRandomSearch`.-* If you prefer to do **instruction optimization only** (i.e. you want to keep your prompt 0-shot), use `MIPROv2` [configured for 0-shot optimization to optimize](/deep-dive/optimizers/miprov2#optimizing-instructions-only-with-miprov2-0-shot).-* If you’re willing to use more inference calls to perform **longer optimization runs** (e.g. 40 trials or more), and have enough data (e.g. 200 examples or more to prevent overfitting) then try `MIPROv2`.-* If you have been able to use one of these with a large LM (e.g., 7B parameters or above) and need a very **efficient program**, finetune a small LM for your task with `BootstrapFinetune`.--## How do I use an optimizer?--They all share this general interface, with some differences in the keyword arguments (hyperparameters). Detailed documentation for key optimizers can be found [here](/deep-dive/optimizers/vfrs), and a full list can be found [here](/cheatsheet).--Let's see this with the most common one, `BootstrapFewShotWithRandomSearch`.--```python-from dspy.teleprompt import BootstrapFewShotWithRandomSearch--# Set up the optimizer: we want to "bootstrap" (i.e., self-generate) 8-shot examples of your program's steps.-# The optimizer will repeat this 10 times (plus some initial attempts) before selecting its best attempt on the devset.-config = dict(max_bootstrapped_demos=4, max_labeled_demos=4, num_candidate_programs=10, num_threads=4)--teleprompter = BootstrapFewShotWithRandomSearch(metric=YOUR_METRIC_HERE, **config)-optimized_program = teleprompter.compile(YOUR_PROGRAM_HERE, trainset=YOUR_TRAINSET_HERE)-```--## Saving and loading optimizer output--After running a program through an optimizer, it's useful to also save it. At a later point, a program can be loaded from a file and used for inference. For this, the `load` and `save` methods can be used.--### Saving a program--```python-optimized_program.save(YOUR_SAVE_PATH)-```--The resulting file is in plain-text JSON format. It contains all the parameters and steps in the source program. You can always read it and see what the optimizer generated.--You can add `save_field_meta` to additionally save the list of fields with the keys, `name`, `field_type`, `description`, and `prefix` with: `optimized_program.save(YOUR_SAVE_PATH, save_field_meta=True).--### Loading a program--To load a program from a file, you can instantiate an object from that class and then call the load method on it.--```python-loaded_program = YOUR_PROGRAM_CLASS()-loaded_program.load(path=YOUR_SAVE_PATH)-```diff --git a/docs/docs/building-blocks/_category_.json b/docs/docs/building-blocks/_category_.jsondeleted file mode 100644index 003f099b2..000000000--- a/docs/docs/building-blocks/_category_.json+++ /dev/null@@ -1,8 +0,0 @@-{- "label": "DSPy Building Blocks",- "position": 3,- "link": {- "type": "generated-index",- "description": "DSPy introduces signatures (to abstract prompts), modules (to abstract prompting techniques), and optimizers that can tune the prompts (or weights) of modules."- }-}diff --git a/docs/docs/deep-dive/assertions.md b/docs/docs/deep-dive/assertions.mdindex edb25013a..858d03780 100644--- a/docs/docs/deep-dive/assertions.md+++ b/docs/docs/deep-dive/assertions.md@@ -4,6 +4,9 @@ sidebar_position: 7# DSPy Assertions+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++Language models (LMs) have transformed how we interact with machine learning, offering vast capabilities in natural language understanding and generation. However, ensuring these models adhere to domain-specific constraints remains a challenge. Despite the growth of techniques like fine-tuning or “prompt engineering”, these approaches are extremely tedious and rely on heavy, manual hand-waving to guide the LMs in adhering to specific constraints. Even DSPy's modularity of programming prompting pipelines lacks mechanisms to effectively and automatically enforce these constraints.To address this, we introduce DSPy Assertions, a feature within the DSPy framework designed to automate the enforcement of computational constraints on LMs. DSPy Assertions empower developers to guide LMs towards desired outcomes with minimal manual intervention, enhancing the reliability, predictability, and correctness of LM outputs.diff --git a/docs/docs/deep-dive/data-handling/built-in-datasets.md b/docs/docs/deep-dive/data-handling/built-in-datasets.mdindex 39786b982..f8046addf 100644--- a/docs/docs/deep-dive/data-handling/built-in-datasets.md+++ b/docs/docs/deep-dive/data-handling/built-in-datasets.md@@ -2,6 +2,7 @@sidebar_position: 2---+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"# Utilizing Built-in Datasetsdiff --git a/docs/docs/deep-dive/data-handling/examples.md b/docs/docs/deep-dive/data-handling/examples.mdindex 33462ffd3..3095d719a 100644--- a/docs/docs/deep-dive/data-handling/examples.md+++ b/docs/docs/deep-dive/data-handling/examples.md@@ -2,6 +2,7 @@sidebar_position: 1---+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"# Examples in DSPydiff --git a/docs/docs/deep-dive/data-handling/loading-custom-data.md b/docs/docs/deep-dive/data-handling/loading-custom-data.mdindex 196d0119b..09a10e6e5 100644--- a/docs/docs/deep-dive/data-handling/loading-custom-data.md+++ b/docs/docs/deep-dive/data-handling/loading-custom-data.md@@ -2,6 +2,8 @@sidebar_position: 3---+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"+# Creating a Custom DatasetWe've seen how to work with with `Example` objects and use the `HotPotQA` class to load the HuggingFace HotPotQA dataset as a list of `Example` objects. But in production, such structured datasets are rare. Instead, you'll find yourself working on a custom dataset and might question: how do I create my own dataset or what format should it be?diff --git a/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientTGI.md b/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientTGI.mdindex 3c136e0de..71580e0c9 100644--- a/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientTGI.md+++ b/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientTGI.md@@ -2,6 +2,9 @@ import AuthorDetails from '@site/src/components/AuthorDetails';# HFClientTGI+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## PrerequisitesDocker must be installed on your system. If you don't have Docker installed, you can get it from [here](https://docs.docker.com/get-docker/).diff --git a/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientVLLM.md b/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientVLLM.mdindex 1d2a4fc93..7b7697b92 100644--- a/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientVLLM.md+++ b/docs/docs/deep-dive/language_model_clients/lm_local_models/HFClientVLLM.md@@ -1,5 +1,8 @@# HFClient vLLM+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## PrerequisitesFollow these steps to set up the vLLM Server:diff --git a/docs/docs/deep-dive/language_model_clients/lm_local_models/LlamaCpp.md b/docs/docs/deep-dive/language_model_clients/lm_local_models/LlamaCpp.mdindex 37de3be6b..a3d8f225c 100644--- a/docs/docs/deep-dive/language_model_clients/lm_local_models/LlamaCpp.md+++ b/docs/docs/deep-dive/language_model_clients/lm_local_models/LlamaCpp.md@@ -1,5 +1,8 @@# LlamaCpp+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## PrerequisitesInstall Llama Cpp Python by following the instructions provided in the [Llama Cpp Python repository](https://github.com/abetlen/llama-cpp-python).diff --git a/docs/docs/deep-dive/language_model_clients/lm_local_models/MLC.md b/docs/docs/deep-dive/language_model_clients/lm_local_models/MLC.mdindex a34f29d56..d926fbfc4 100644--- a/docs/docs/deep-dive/language_model_clients/lm_local_models/MLC.md+++ b/docs/docs/deep-dive/language_model_clients/lm_local_models/MLC.md@@ -1,5 +1,8 @@# ChatModuleClient+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## Prerequisites1. Install the required packages using the following commands:diff --git a/docs/docs/deep-dive/language_model_clients/lm_local_models/Ollama.md b/docs/docs/deep-dive/language_model_clients/lm_local_models/Ollama.mdindex a9d7aa3d3..2f167b518 100644--- a/docs/docs/deep-dive/language_model_clients/lm_local_models/Ollama.md+++ b/docs/docs/deep-dive/language_model_clients/lm_local_models/Ollama.md@@ -1,5 +1,7 @@# OllamaLocal+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"+!!! noteAdapted from documentation provided by https://github.com/insopdiff --git a/docs/docs/deep-dive/language_model_clients/lm_local_models/TensorRTLLM.md b/docs/docs/deep-dive/language_model_clients/lm_local_models/TensorRTLLM.mdindex 99045a18e..d81a1727f 100644--- a/docs/docs/deep-dive/language_model_clients/lm_local_models/TensorRTLLM.md+++ b/docs/docs/deep-dive/language_model_clients/lm_local_models/TensorRTLLM.md@@ -1,5 +1,7 @@# TensorRTModel+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"+TensorRT LLM by Nvidia happens to be one of the most optimized inference engines to run open-source Large Language Models locally or in production.### Prerequisitesdiff --git a/docs/docs/deep-dive/modules/chain-of-thought-with-hint.md b/docs/docs/deep-dive/modules/chain-of-thought-with-hint.mdindex e4c160a4d..43845f920 100644--- a/docs/docs/deep-dive/modules/chain-of-thought-with-hint.md+++ b/docs/docs/deep-dive/modules/chain-of-thought-with-hint.md@@ -4,6 +4,8 @@ sidebar_position: 2# ChainOfThoughtWithHint+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"+This class builds upon the `ChainOfThought` class by introducing an additional input field to provide hints for reasoning. The inclusion of a hint allows for a more directed problem-solving process, which can be especially useful in complex scenarios.diff --git a/docs/docs/deep-dive/modules/chain-of-thought.md b/docs/docs/deep-dive/modules/chain-of-thought.mdindex 1efb0fe94..f5bacee50 100644--- a/docs/docs/deep-dive/modules/chain-of-thought.md+++ b/docs/docs/deep-dive/modules/chain-of-thought.md@@ -1,5 +1,8 @@# dspy.ChainOfThought+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++### ConstructorThe constructor initializes the `ChainOfThought` class and sets up its attributes. It inherits from the `Predict` class and adds specific functionality for chain of thought processing.diff --git a/docs/docs/deep-dive/modules/guide.md b/docs/docs/deep-dive/modules/guide.mdindex f0e6beb33..65a61f3e9 100644--- a/docs/docs/deep-dive/modules/guide.md+++ b/docs/docs/deep-dive/modules/guide.md@@ -1,5 +1,8 @@# DSPy Modules+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++[<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/docs/guides/modules.ipynb)## Quick Recapdiff --git a/docs/docs/deep-dive/modules/multi-chain-comparison.md b/docs/docs/deep-dive/modules/multi-chain-comparison.mdindex 690708fea..6c142d282 100644--- a/docs/docs/deep-dive/modules/multi-chain-comparison.md+++ b/docs/docs/deep-dive/modules/multi-chain-comparison.md@@ -1,5 +1,8 @@# dspy.MultiChainComparison+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## ConstructorThe constructor initializes the `MultiChainComparison` class and sets up its attributes. It inherits from the `Predict` class and adds specific functionality for multiple chain comparisons.diff --git a/docs/docs/deep-dive/modules/predict.md b/docs/docs/deep-dive/modules/predict.mdindex b5169ea12..259859f9c 100644--- a/docs/docs/deep-dive/modules/predict.md+++ b/docs/docs/deep-dive/modules/predict.md@@ -1,5 +1,8 @@# dspy.Predict+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## ConstructorThe constructor initializes the `Predict` class and sets up its attributes, taking in the `signature` and additional config options. If the `signature` is a string, it processes the input and output fields, generates instructions, and creates a template for the specified `signature` type.diff --git a/docs/docs/deep-dive/modules/program-of-thought.md b/docs/docs/deep-dive/modules/program-of-thought.mdindex f78cbc833..52fa7ab98 100644--- a/docs/docs/deep-dive/modules/program-of-thought.md+++ b/docs/docs/deep-dive/modules/program-of-thought.md@@ -4,6 +4,9 @@ sidebar_position: 2# Program of Thought+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## BackgroundDSPy supports the Program of Thought (PoT) prompting technique, integrating an advanced module capable of problem-solving with program execution capabilities. PoT builds upon Chain of Thought by translating reasoning steps into executable programming language statements through iterative refinement. This enhances the accuracy of the output and self-corrects errors within the generated code. Upon completing these iterations, PoT delegates execution to a program interpreter. Currently, this class supports the generation and execution of Python code.diff --git a/docs/docs/deep-dive/modules/react.md b/docs/docs/deep-dive/modules/react.mdindex e5e463574..ab4041368 100644--- a/docs/docs/deep-dive/modules/react.md+++ b/docs/docs/deep-dive/modules/react.md@@ -4,6 +4,9 @@ sidebar_position: 2# ReAct+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## BackgroundDSPy supports ReAct, an LLM agent designed to tackle complex tasks in an interactive fashion. ReAct is composed of an iterative loop of interpretation, decision and action-based activities ("Thought, Action, and Observation") based on an evolving set of input and output fields. Through this real-time iterative approach, the ReAct agent can both analyze and adapt to its responses over time as new information becomes available.diff --git a/docs/docs/deep-dive/modules/retrieve.md b/docs/docs/deep-dive/modules/retrieve.mdindex 121300275..f69705367 100644--- a/docs/docs/deep-dive/modules/retrieve.md+++ b/docs/docs/deep-dive/modules/retrieve.md@@ -1,5 +1,7 @@# Retrieve+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"+## BackgroundDSPy supports retrieval through the Retrieve module that serves to process user queries and output relevant passages from retrieval corpuses. This module ties in with the DSPy-supported Retrieval Model Clients which are retrieval servers that users can utilize to retrieve information for information retrieval tasks.diff --git a/docs/docs/deep-dive/optimizers/BootstrapFinetune.md b/docs/docs/deep-dive/optimizers/BootstrapFinetune.mdindex 941848523..7aba6ec9b 100644--- a/docs/docs/deep-dive/optimizers/BootstrapFinetune.md+++ b/docs/docs/deep-dive/optimizers/BootstrapFinetune.md@@ -4,6 +4,9 @@ sidebar_position: 4# teleprompt.BootstrapFinetune+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++### Constructor### `__init__(self, metric=None, teacher_settings={}, multitask=True)`diff --git a/docs/docs/deep-dive/optimizers/Ensemble.md b/docs/docs/deep-dive/optimizers/Ensemble.mdindex eb9ddb61b..a6282040a 100644--- a/docs/docs/deep-dive/optimizers/Ensemble.md+++ b/docs/docs/deep-dive/optimizers/Ensemble.md@@ -4,6 +4,9 @@ sidebar_position: 4# teleprompt.Ensemble+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++### ConstructorThe constructor initializes the `Ensemble` class and sets up its attributes. This teleprompter is designed to create ensembled versions of multiple programs, reducing various outputs from different programs into a single output.diff --git a/docs/docs/deep-dive/optimizers/LabeledFewShot.md b/docs/docs/deep-dive/optimizers/LabeledFewShot.mdindex c815a8d18..00cd7ce8e 100644--- a/docs/docs/deep-dive/optimizers/LabeledFewShot.md+++ b/docs/docs/deep-dive/optimizers/LabeledFewShot.md@@ -4,6 +4,9 @@ sidebar_position: 2# LabeledFewShot+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++### ConstructorThe constructor initializes the `LabeledFewShot` class and sets up its attributes, particularly defining `k` number of samples to be used by the predictor.diff --git a/docs/docs/deep-dive/optimizers/bfrs.md b/docs/docs/deep-dive/optimizers/bfrs.mdindex c15004af1..780c013cd 100644--- a/docs/docs/deep-dive/optimizers/bfrs.md+++ b/docs/docs/deep-dive/optimizers/bfrs.md@@ -4,6 +4,9 @@ sidebar_position: 3# BootstrapFewShotWithRandomSearch+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++`BootstrapFewShotWithRandomSearch` is an teleprompter that extends `BootstrapFewShot` by incorporating a random search strategy to optimize the selection of few-shot examples. This teleprompter is useful when you want to explore a variety of example combinations to find the best examples for your model.## Usagediff --git a/docs/docs/deep-dive/optimizers/bootstrap-fewshot.md b/docs/docs/deep-dive/optimizers/bootstrap-fewshot.mdindex 2f963c717..fe435018f 100644--- a/docs/docs/deep-dive/optimizers/bootstrap-fewshot.md+++ b/docs/docs/deep-dive/optimizers/bootstrap-fewshot.md@@ -4,6 +4,9 @@ sidebar_position: 1# BootstrapFewShot+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++When compiling a DSPy program, we generally invoke an optimizer that takes the program, a training set, and a metric and returns a new optimized program. Different optimizers apply different strategies for optimization. This family of optimizers is focused on optimizing the few shot examples. Let's take an example of a Sample pipeline and see how we can use this optimizer to optimize it.## Setting up a Sample Pipelinediff --git a/docs/docs/deep-dive/optimizers/copro.md b/docs/docs/deep-dive/optimizers/copro.mdindex c604a3260..f8c5ebbbf 100644--- a/docs/docs/deep-dive/optimizers/copro.md+++ b/docs/docs/deep-dive/optimizers/copro.md@@ -4,6 +4,9 @@ sidebar_position: 5# `COPRO`+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++`COPRO` which aims to improve the output prefixes and instruction of the signatures in a module in a zero/few shot setting. This teleprompter is especially beneficial for fine-tuning the prompt for language models and ensure they perform tasks more effectively, all from a vague and unrefined prompt.## Setting up a Sample Pipelinediff --git a/docs/docs/deep-dive/optimizers/miprov2.md b/docs/docs/deep-dive/optimizers/miprov2.mdindex f0ec47522..df6a03e26 100644--- a/docs/docs/deep-dive/optimizers/miprov2.md+++ b/docs/docs/deep-dive/optimizers/miprov2.md@@ -4,6 +4,9 @@ sidebar_position: 6# `MIPROv2` Optimizer+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## Overview`MIPROv2` (<u>M</u>ultiprompt <u>I</u>nstruction <u>PR</u>oposal <u>O</u>ptimizer Version 2) is an prompt optimizer capable of optimizing both instructions and few-shot examples jointly. It does this by bootstrapping few-shot example candidates, proposing instructions grounded in different dynamics of the task, and finding an optimized combination of these options using Bayesian Optimization. It can be used for optimizing few-shot examples & instructions jointly, or just instructions for 0-shot optimization.diff --git a/docs/docs/deep-dive/signature/_category_.json b/docs/docs/deep-dive/signature/_category_.jsondeleted file mode 100644index b7ed58d84..000000000--- a/docs/docs/deep-dive/signature/_category_.json+++ /dev/null@@ -1,8 +0,0 @@-{- "label": "Signatures",- "position": 2,- "link": {- "type": "generated-index",- "description": "Signatures in DSPy"- }-}\ No newline at end of filediff --git a/docs/docs/deep-dive/signature/understanding-signatures.md b/docs/docs/deep-dive/signature/understanding-signatures.mddeleted file mode 100644index c3c028814..000000000--- a/docs/docs/deep-dive/signature/understanding-signatures.md+++ /dev/null@@ -1,119 +0,0 @@-----sidebar_position: 1------# Understanding Signatures--A DSPy Signature is the most basic form of task description which simply requires inputs and outputs and optionally, a small description about them and the task too.--There are 2 ways to define a Signature: **Inline** and **Class-Based**. But before diving into creating signatures, let's understand what a signature is and why we need it.--## What is a Signature?--In the typical LLM pipeline, you'll have two key components at work i.e. an LLM and a prompt. In DSPy, we have an LLM configured at the beginning of any DSPy script via the LM(Language Model - which is shown in the next blog) and a prompt defined via **Signatures**.--A Signature is usually composed of 2 essential components: **Input Fields** and **Output Fields**. You can optionally pass an instruction defining more robust requirements of your task. An **Input Field** is an attribute of Signature that defines an input to the prompt and an **Output Field** is an attribute of Signature that defines an output of the prompt received from an LLM call. Let's understand this by an example.----Let's think of a basic Question-Answer task where the question serves as an input to the LLM from which you receive an answer response. We directly map this in DSPy as the question serves as the Signature's **Input Field** and the answer as the Signature's **Output Field** .--Now that we understand the components of a Signature, let's see how we can declare a signature and what a prompt for that signature looks like.--## Inline Method--DSPy offers an intuitive, simple approach for defining tasks: simply state the inputs and outputs to convey the task in its simplest form. For example, if your input is **question** and output is **answer**, it should be clear that the task is a Question-Answer task. If your inputs are **context** and **question** and outputs are **answer** and **reason**, this should imply some form of Chain-Of-Thought prompting, potentially within a RAG pipeline.--Inspired by this simplicity, DSPy Signatures mirrors an Einops-like abstract manner:--```text-input_field_1,input_field_2,input_field_3...->output_field_1,output_field_2,output_field_3...-```--**Input Fields** of the Signature are declared on the left side of `->` with the **Output Fields** on the right side. So let's go ahead and define DSPy signatures for the QA and RAG tasks:--```text-QA Task: question->answer-RAG Task: context,question->answer,rationale-```--This simplistic naming of the fields is essential for the LLM to understand the nature of inputs and outputs, reducing sensitivity and ensuring clarity for expected inputs and generations.--However, this barebones signature may not provide a clear picture for how the model should approach the task, and to meet these needs, DSPy modules offer simplistic yet robust instructional templates that integrate the Signatures.-Let's take a deeper look at the prompt constructed by DSPy to understand it better when used within a `dspy.Predict` module as `dspy.Predict(question->answer)`:--```-Given the fields `question`, produce the fields `answer`.-------Follow the following format.--Question: ${question}-Answer: ${answer}-------Question:-```--As you can see, DSPy populates the instruction `Given the fields ``question``, produce the fields ``answer``.` to define the task and provides instructions for the prompt format. And this format is pretty standard for any Signature you create as we can see in this prompting setup for RAG:----Now these instructional templates are well defined for their respective prompting techniques (CoT, ProgramOfThought, ReAct), leaving the user only having to define their task's Signature input and outputs with the rest handled by the DSPy modules library!--However, it would be nice to give more instructions beyond the simplistic in-line signature and for this, we turn to class-based signatures.--## Class Based Method--A Signature class comprises of three things:--* **Task Description/Instruction:** We define in the signature class docstring.-* **Inputs Field:** We define these as `dspy.InputField()`.-* **Outputs Field:** We define these as `dspy.OutputField()`.--```python-class BasicQA(dspy.Signature):- """Answer questions with short factoid answers."""-- question = dspy.InputField()- answer = dspy.OutputField(desc="often between 1 and 5 words", prefix="Question's Answer:")-```--The I/O Fields take 3 inputs: `desc`, `prefix` and `format`. `desc` is the description to the input, `prefix` is the placeholder text of the field in the prompt(one that has been `${field_name}` until now) and `format` which is a method that'll define how to handle non-string inputs. If the input to field is a list rather than a string, we can specify this through `format`.--Both `InputField` and `OutputField` are similar in implementation as well:--```python-class InputField(Field):- def __init__(self, *, prefix=None, desc=None, format=None):- super().__init__(prefix=prefix, desc=desc, input=True, format=format)--class OutputField(Field):- def __init__(self, *, prefix=None, desc=None, format=None):- super().__init__(prefix=prefix, desc=desc, input=False, format=format)-```--Let's take a look at how a prompt for the class based signature looks like:--```text-Answer questions with short factoid answers.-------Follow the following format.--Question: ${question}-Question's Answer: often between 1 and 5 words-------Question:-```--As you can see, the instruction is more well-defined by our task's instruction in the docstring. The prefix and description for the `answer` field reflects our definitions. This ensures a more refined prompt structure, giving the user more control on defining its contents per task requirements.----***diff --git a/docs/docs/deep-dive/typed_predictors/_category_.json b/docs/docs/deep-dive/typed_predictors/_category_.jsondeleted file mode 100644index 2b9a4e2f0..000000000--- a/docs/docs/deep-dive/typed_predictors/_category_.json+++ /dev/null@@ -1,8 +0,0 @@-{- "label": "Typed Predictors",- "position": 4,- "link": {- "type": "generated-index",- "description": "Typed Predictors in DSPy"- }-}\ No newline at end of filediff --git a/docs/docs/deep-dive/typed_predictors/functional_typed_predictors.md b/docs/docs/deep-dive/typed_predictors/functional_typed_predictors.mddeleted file mode 100644index 9a8f79ae9..000000000--- a/docs/docs/deep-dive/typed_predictors/functional_typed_predictors.md+++ /dev/null@@ -1,7 +0,0 @@-# Functional Typed Predictors--## Typed Predictors as Decorators--## Functional Typed Predictors in `dspy.Module`--## How Functional Typed Predictors work?\ No newline at end of filediff --git a/docs/docs/deep-dive/typed_predictors/understanding_predictors.md b/docs/docs/deep-dive/typed_predictors/understanding_predictors.mddeleted file mode 100644index f0ad17354..000000000--- a/docs/docs/deep-dive/typed_predictors/understanding_predictors.md+++ /dev/null@@ -1,9 +0,0 @@-# Understanding Typed Predictors--## Why use a Typed Predictor?--## How to use a Typed Predictor?--## Prompt of Typed Predictors--## How Typed Predictors work?\ No newline at end of filediff --git a/docs/docs/building-blocks/8-typed_predictors.md b/docs/docs/learn/8-typed_predictors.mdsimilarity index 100%rename from docs/docs/building-blocks/8-typed_predictors.mdrename to docs/docs/learn/8-typed_predictors.mddiff --git a/docs/docs/building-blocks/4-data.md b/docs/docs/learn/evaluation/data.mdsimilarity index 79%rename from docs/docs/building-blocks/4-data.mdrename to docs/docs/learn/evaluation/data.mdindex 577ce33ad..eba40f77e 100644--- a/docs/docs/building-blocks/4-data.md+++ b/docs/docs/learn/evaluation/data.md@@ -4,19 +4,8 @@ sidebar_position: 5# Data-DSPy is a machine learning framework, so working in it involves training sets, development sets, and test sets.+DSPy is a machine learning framework, so working in it involves training sets, development sets, and test sets. For each example in your data, we distinguish typically between three types of values: the inputs, the intermediate labels, and the final label. You can use DSPy effectively without any intermediate or final labels, but you will need at least a few example inputs.-For each example in your data, we distinguish typically between three types of values: the inputs, the intermediate labels, and the final label. You can use DSPy effectively without any intermediate or final labels, but you will need at least a few example inputs.--## How much data do I need and how do I collect data for my task?--Concretely, you can use DSPy optimizers usefully with as few as 10 example inputs, but having 50-100 examples (or even better, 300-500 examples) goes a long way.--How can you get examples like these? If your task is extremely unusual, please invest in preparing ~10 examples by hand. Often times, depending on your metric below, you just need inputs and not labels, so it's not that hard.--However, chances are that your task is not actually that unique. You can almost always find somewhat adjacent datasets on, say, HuggingFace datasets or other forms of data that you can leverage here.--If there's data whose licenses are permissive enough, we suggest you use them. Otherwise, you can also start using/deploying/demoing your system and collect some initial data that way.## DSPy `Example` objects@@ -87,7 +76,7 @@ Example object with Input fields only: Example({'article': 'This is an article.'Example object with Non-Input fields only: Example({'summary': 'This is a summary.'}) (input_keys=None)```-## Loading Dataset from sources+<!-- ## Loading Dataset from sourcesOne of the most convenient way to import datasets in DSPy is by using `DataLoader`. The first step is to declare an object, this object can then be used to call utilities to load datasets in different formats:@@ -148,4 +137,4 @@ testset = train_split[:75]trainset = train_split[75:]```-The way you load a HuggingFace dataset using `load_dataset` is exactly how you load data it via `from_huggingface` as well. This includes passing specific splits, subsplits, read instructions, etc. For code snippets, you can refer to the [cheatsheet snippets](/cheatsheet/#dspy-dataloaders) for loading from HF.\ No newline at end of file+The way you load a HuggingFace dataset using `load_dataset` is exactly how you load data it via `from_huggingface` as well. This includes passing specific splits, subsplits, read instructions, etc. For code snippets, you can refer to the [cheatsheet snippets](/cheatsheet/#dspy-dataloaders) for loading from HF. -->\ No newline at end of filediff --git a/docs/docs/building-blocks/5-metrics.md b/docs/docs/learn/evaluation/metrics.mdsimilarity index 89%rename from docs/docs/building-blocks/5-metrics.mdrename to docs/docs/learn/evaluation/metrics.mdindex 377e7d78e..9e1f742f1 100644--- a/docs/docs/building-blocks/5-metrics.md+++ b/docs/docs/learn/evaluation/metrics.md@@ -93,25 +93,22 @@ class Assess(dspy.Signature):assessed_text = dspy.InputField()assessment_question = dspy.InputField()- assessment_answer = dspy.OutputField(desc="Yes or No")+ assessment_answer: bool = dspy.OutputField()```-For example, below is a simple metric that uses GPT-4-turbo to check if a generated tweet (1) answers a given question correctly and (2) whether it's also engaging. We also check that (3) `len(tweet) <= 280` characters.+For example, below is a simple metric that checks a generated tweet (1) answers a given question correctly and (2) whether it's also engaging. We also check that (3) `len(tweet) <= 280` characters.```python-gpt4T = dspy.OpenAI(model='gpt-4-1106-preview', max_tokens=1000, model_type='chat')-def metric(gold, pred, trace=None):question, answer, tweet = gold.question, gold.answer, pred.outputengaging = "Does the assessed text make for a self-contained, engaging tweet?"correct = f"The text should answer `{question}` with `{answer}`. Does the assessed text contain this answer?"- with dspy.context(lm=gpt4T):- correct = dspy.Predict(Assess)(assessed_text=tweet, assessment_question=correct)- engaging = dspy.Predict(Assess)(assessed_text=tweet, assessment_question=engaging)+ correct = dspy.Predict(Assess)(assessed_text=tweet, assessment_question=correct)+ engaging = dspy.Predict(Assess)(assessed_text=tweet, assessment_question=engaging)- correct, engaging = [m.assessment_answer.lower() == 'yes' for m in [correct, engaging]]+ correct, engaging = [m.assessment_answer for m in [correct, engaging]]score = (correct + engaging) if correct and (len(tweet) <= 280) else 0if trace is not None: return score >= 2diff --git a/docs/docs/learn/evaluation/overview.md b/docs/docs/learn/evaluation/overview.mdnew file mode 100644index 000000000..e435b0138--- /dev/null+++ b/docs/docs/learn/evaluation/overview.md@@ -0,0 +1,16 @@+---+sidebar_position: 1+---++# Evaluation in DSPy++Once you have a system that works reasonably well, it's time to **collect an initial development set**. This will help you refine your system more systematically. Even 20 input examples of your task can be useful, though 200 goes a long way. You could prepare a few examples by hand: depending on your _metric_, you either just need inputs and no labels at all, or you need inputs and the _final_ outputs of your system. (You almost never need labels for the intermediate steps in your program in DSPy.) Alternatively, you can probably find datasets that are adjacent to your task on, say, HuggingFace datasets or in a naturally occuring source like StackExchange. If there's data whose licenses are permissive enough, we suggest you use them. Otherwise, you can also start deploying a demo of your system and collect initial data that way.++Next, you should **define your DSPy metric**. What makes outputs from your system good or bad? Invest in defining metrics and improving them incrementally over time. It's really hard to consistently improve what you aren't able to define. A metric is just a function that will take examples from your data and take the output of your system, and return a score that quantifies how good the output is. For simple tasks, this could be just "accuracy" or "exact match" or "F1 score". This may be the case for simple classification or short-form QA tasks. For most applications, your system will produce long-form outputs, so your metric will be a smaller DSPy program that checks multiple properties of the output. Getting this right on the first try is unlikely, but you should start with something simple and iterate.++Now that you have some data and a metric, run development evaluations on your pipeline designs to understand their tradeoffs. Look at the outputs and the metric scores. This will probably allow you to spot any major issues, and it will define a baseline for your next steps.+++??? "If your metric is itself a DSPy program..."+ If your metric is itself a DSPy program, a powerful way to iterate is to optimize your metric itself. That's usually easy because the output of the metric is usually a simple value (e.g., a score out of 5), so the metric's metric is easy to define and optimize by collecting a few examples.+diff --git a/docs/docs/building-blocks/figures/teleprompter-classes.png b/docs/docs/learn/figures/teleprompter-classes.pngsimilarity index 100%rename from docs/docs/building-blocks/figures/teleprompter-classes.pngrename to docs/docs/learn/figures/teleprompter-classes.pngdiff --git a/docs/docs/learn/index.md b/docs/docs/learn/index.mdnew file mode 100644index 000000000..5b61f76de--- /dev/null+++ b/docs/docs/learn/index.md@@ -0,0 +1,16 @@+---+sidebar_position: 1+---++# Learning DSPy: An Overview++As a framework, DSPy exposes a very small API that you can learn quickly. However, using it for novel tasks involves the more open-ended journey of composing its tools and design patterns to build _your_ modular AI systems quickly and to optimize them for _your_ objectives. This can be done along three different levels of system development, presented below.+++1) **Programming in DSPy.** This is the stage in which your concern is to define your task, its constraints, explore a few examples of it, and use that to inform your initial pipeline design.++2) **Evaluation in DSPy.** Once you have a system that works reasonably well, you collect an initial small development set, define your DSPy metric, and use these to iterate on your system more systematically.++3) **Optimization in DSPy.** Once you have a system and a way to evaluate it, you can start to use DSPy optimizers to tune the prompts or weights in your program.++We typically suggest that you learn and apply DSPy in this order. For example, it's unproductive to launch optimization runs using a poorly-design program or a bad metric.\ No newline at end of filediff --git a/docs/docs/learn/optimization/optimizers.md b/docs/docs/learn/optimization/optimizers.mdnew file mode 100644index 000000000..d753bb3e7--- /dev/null+++ b/docs/docs/learn/optimization/optimizers.md@@ -0,0 +1,223 @@+---+sidebar_position: 1+---++# DSPy Optimizers (formerly Teleprompters)+++A **DSPy optimizer** is an algorithm that can tune the parameters of a DSPy program (i.e., the prompts and/or the LM weights) to maximize the metrics you specify, like accuracy.+++A typical DSPy optimizer takes three things:++- Your **DSPy program**. This may be a single module (e.g., `dspy.Predict`) or a complex multi-module program.++- Your **metric**. This is a function that evaluates the output of your program, and assigns it a score (higher is better).++- A few **training inputs**. This may be very small (i.e., only 5 or 10 examples) and incomplete (only inputs to your program, without any labels).++If you happen to have a lot of data, DSPy can leverage that. But you can start small and get strong results.++**Note:** Formerly called teleprompters. We are making an official name update, which will be reflected throughout the library and documentation.+++## What does a DSPy Optimizer tune? How does it tune them?++Different optimizers in DSPy will tune your program's quality by **synthesizing good few-shot examples** for every module, like `dspy.BootstrapRS`,<sup>[1](https://arxiv.org/abs/2310.03714)</sup> **proposing and intelligently exploring better natural-language instructions** for every prompt, like `dspy.MIPROv2`,<sup>[2](https://arxiv.org/abs/2406.11695)</sup> and **building datasets for your modules and using them to finetune the LM weights** in your system, like `dspy.BootstrapFinetune`.<sup>[3](https://arxiv.org/abs/2407.10930)</sup>++??? "What's an example of a DSPy optimizer? How do different optimizers work?"++ Take the `dspy.MIPROv2` optimizer as an example. First, MIPRO starts with the **bootstrapping stage**. It takes your program, which may be unoptimized at this point, and runs it many times across different inputs to collect traces of input/output behavior for each one of your modules. It filters these traces to keep only those that appear in trajectories scored highly by your metric. Second, MIPRO enters its **grounded proposal stage**. It previews your DSPy program's code, your data, and traces from running your program, and uses them to draft many potential instructions for every prompt in your program. Third, MIPRO launches the **discrete search stage**. It samples mini-batches from your training set, proposes a combination of instructions and traces to use for constructing every prompt in the pipeline, and evaluates the candidate program on the mini-batch. Using the resulting score, MIPRO updates a surrogate model that helps the proposals get better over time.++ One thing that makes DSPy optimizers so powerful is that they can be composed. You can run `dspy.MIPROv2` and use the produced program as an input to `dspy.MIPROv2` again or, say, to `dspy.BootstrapFinetune` to get better results. This is partly the essence of `dspy.BetterTogether`. Alternatively, you can run the optimizer and then extract the top-5 candidate programs and build a `dspy.Ensemble` of them. This allows you to scale _inference-time compute_ (e.g., ensembles) as well as DSPy's unique _pre-inference time compute_ (i.e., optimization budget) in highly systematic ways.++++## What DSPy Optimizers are currently available?++Optimizers can be accessed via `from dspy.teleprompt import *`.++### Automatic Few-Shot Learning++These optimizers extend the signature by automatically generating and including **optimized** examples within the prompt sent to the model, implementing few-shot learning.++1. **`LabeledFewShot`**: Simply constructs few-shot examples (demos) from provided labeled input and output data points. Requires `k` (number of examples for the prompt) and `trainset` to randomly select `k` examples from.++2. **`BootstrapFewShot`**: Uses a `teacher` module (which defaults to your program) to generate complete demonstrations for every stage of your program, along with labeled examples in `trainset`. Parameters include `max_labeled_demos` (the number of demonstrations randomly selected from the `trainset`) and `max_bootstrapped_demos` (the number of additional examples generated by the `teacher`). The bootstrapping process employs the metric to validate demonstrations, including only those that pass the metric in the "compiled" prompt. Advanced: Supports using a `teacher` program that is a *different* DSPy program that has compatible structure, for harder tasks.++3. [**`BootstrapFewShotWithRandomSearch`**](/deep-dive/optimizers/bootstrap-fewshot): Applies `BootstrapFewShot` several times with random search over generated demonstrations, and selects the best program over the optimization. Parameters mirror those of `BootstrapFewShot`, with the addition of `num_candidate_programs`, which specifies the number of random programs evaluated over the optimization, including candidates of the uncompiled program, `LabeledFewShot` optimized program, `BootstrapFewShot` compiled program with unshuffled examples and `num_candidate_programs` of `BootstrapFewShot` compiled programs with randomized example sets.++4. **`KNNFewShot`**. Uses k-Nearest Neighbors algorithm to find the nearest training example demonstrations for a given input example. These nearest neighbor demonstrations are then used as the trainset for the BootstrapFewShot optimization process. See [this notebook](https://github.com/stanfordnlp/dspy/blob/main/examples/knn.ipynb) for an example.+++### Automatic Instruction Optimization++These optimizers produce optimal instructions for the prompt and, in the case of MIPROv2 can also optimize the set of few-shot demonstrations.++5. [**`COPRO`**](/deep-dive/optimizers/copro): Generates and refines new instructions for each step, and optimizes them with coordinate ascent (hill-climbing using the metric function and the `trainset`). Parameters include `depth` which is the number of iterations of prompt improvement the optimizer runs over.++6. [**`MIPROv2`**](/deep-dive/optimizers/miprov2): Generates instructions *and* few-shot examples in each step. The instruction generation is data-aware and demonstration-aware. Uses Bayesian Optimization to effectively search over the space of generation instructions/demonstrations across your modules.+++### Automatic Finetuning++This optimizer is used to fine-tune the underlying LLM(s).++7. **`BootstrapFinetune`**: Distills a prompt-based DSPy program into weight updates. The output is a DSPy program that has the same steps, but where each step is conducted by a finetuned model instead of a prompted LM.+++### Program Transformations++8. **`Ensemble`**: Ensembles a set of DSPy programs and either uses the full set or randomly samples a subset into a single program.+++## Which optimizer should I use?++Ultimately, finding the ‘right’ optimizer to use & the best configuration for your task will require experimentation. Success in DSPy is still an iterative process - getting the best performance on your task will require you to explore and iterate.++That being said, here's the general guidance on getting started:++- If you have **very few examples** (around 10), start with `BootstrapFewShot`.+- If you have **more data** (50 examples or more), try `BootstrapFewShotWithRandomSearch`.+- If you prefer to do **instruction optimization only** (i.e. you want to keep your prompt 0-shot), use `MIPROv2` [configured for 0-shot optimization to optimize](/deep-dive/optimizers/miprov2#optimizing-instructions-only-with-miprov2-0-shot).+- If you’re willing to use more inference calls to perform **longer optimization runs** (e.g. 40 trials or more), and have enough data (e.g. 200 examples or more to prevent overfitting) then try `MIPROv2`.+- If you have been able to use one of these with a large LM (e.g., 7B parameters or above) and need a very **efficient program**, finetune a small LM for your task with `BootstrapFinetune`.++## How do I use an optimizer?++They all share this general interface, with some differences in the keyword arguments (hyperparameters). Detailed documentation for key optimizers can be found [here](/deep-dive/optimizers/vfrs), and a full list can be found [here](/cheatsheet).++Let's see this with the most common one, `BootstrapFewShotWithRandomSearch`.++```python+from dspy.teleprompt import BootstrapFewShotWithRandomSearch++# Set up the optimizer: we want to "bootstrap" (i.e., self-generate) 8-shot examples of your program's steps.+# The optimizer will repeat this 10 times (plus some initial attempts) before selecting its best attempt on the devset.+config = dict(max_bootstrapped_demos=4, max_labeled_demos=4, num_candidate_programs=10, num_threads=4)++teleprompter = BootstrapFewShotWithRandomSearch(metric=YOUR_METRIC_HERE, **config)+optimized_program = teleprompter.compile(YOUR_PROGRAM_HERE, trainset=YOUR_TRAINSET_HERE)+```+++!!! info "Getting Started III: Optimizing the LM prompts or weights in DSPy programs"+ A typical simple optimization run costs on the order of $2 USD and takes around ten minutes, but be careful when running optimizers with very large LMs or very large datasets.+ Optimizer runs can cost as little as a few cents or up to tens of dollars, depending on your LM, dataset, and configuration.++ === "Optimizing prompts for a ReAct agent"+ This is a minimal but fully runnable example of setting up a `dspy.ReAct` agent that answers questions via+ search from Wikipedia and then optimizing it using `dspy.MIPROv2` in the cheap `light` mode on 500+ question-answer pairs sampled from the `HotPotQA` dataset.++ ```python linenums="1"+ import dspy+ from dspy.datasets import HotPotQA++ dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))++ def search(query: str) -> list[str]:+ """Retrieves abstracts from Wikipedia."""+ results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)+ return [x['text'] for x in results]++ trainset = [x.with_inputs('question') for x in HotPotQA(train_seed=2024, train_size=500).train]+ react = dspy.ReAct("question -> answer", tools=[search])++ tp = dspy.MIPROv2(metric=dspy.evaluate.answer_exact_match, auto="light", num_threads=24)+ optimized_react = tp.compile(react, trainset=trainset)+ ```++ An informal run similar to this on DSPy 2.5.29 raises ReAct's score from 24% to 51%.++ === "Optimizing prompts for RAG"+ Given a retrieval index to `search`, your favorite `dspy.LM`, and a small `trainset` of questions and ground-truth responses, the following code snippet can optimize your RAG system with long outputs against the built-in `dspy.SemanticF1` metric, which is implemented as a DSPy module.++ ```python linenums="1"+ class RAG(dspy.Module):+ def __init__(self, num_docs=5):+ self.num_docs = num_docs+ self.respond = dspy.ChainOfThought('context, question -> response')++ def forward(self, question):+ context = search(question, k=self.num_docs) # not defined in this snippet, see link above+ return self.respond(context=context, question=question)++ tp = dspy.MIPROv2(metric=dspy.SemanticF1(), auto="medium", num_threads=24)+ optimized_rag = tp.compile(RAG(), trainset=trainset, max_bootstrapped_demos=2, max_labeled_demos=2)+ ```++ For a complete RAG example that you can run, start this [tutorial](http://127.0.0.1:8000/quick-start/getting-started-01/). It improves the quality of a RAG system over a subset of StackExchange communities from 53% to 61%.++ === "Optimizing weights for Classification"+ This is a minimal but fully runnable example of setting up a `dspy.ChainOfThought` module that classifies+ short texts into one of 77 banking labels and then using `dspy.BootstrapFinetune` with 2000 text-label pairs+ from the `Banking77` to finetune the weights of GPT-4o-mini for this task. We use the variant+ `dspy.ChainOfThoughtWithHint`, which takes an optional `hint` at bootstrapping time, to maximize the utility of+ the training data. Naturally, hints are not available at test time.++ <details><summary>Click to show dataset setup code.</summary>++ ```python linenums="1"+ import random+ from typing import Literal+ from dspy.datasets import DataLoader+ from datasets import load_dataset++ # Load the Banking77 dataset.+ CLASSES = load_dataset("PolyAI/banking77", split="train", trust_remote_code=True).features['label'].names+ kwargs = dict(fields=("text", "label"), input_keys=("text",), split="train", trust_remote_code=True)++ # Load the first 2000 examples from the dataset, and assign a hint to each *training* example.+ trainset = [+ dspy.Example(x, hint=CLASSES[x.label], label=CLASSES[x.label]).with_inputs("text", "hint")+ for x in DataLoader().from_huggingface(dataset_name="PolyAI/banking77", **kwargs)[:2000]+ ]+ random.Random(0).shuffle(trainset)+ ```+ </details>++ ```python linenums="1"+ import dspy+ dspy.configure(lm=dspy.LM('gpt-4o-mini-2024-07-18'))++ # Define the DSPy module for classification. It will use the hint at training time, if available.+ signature = dspy.Signature("text -> label").with_updated_fields('label', type_=Literal[tuple(CLASSES)])+ classify = dspy.ChainOfThoughtWithHint(signature)++ # Optimize via BootstrapFinetune.+ optimizer = dspy.BootstrapFinetune(metric=(lambda x, y, trace=None: x.label == y.label), num_threads=24)+ optimized = optimizer.compile(classify, trainset=trainset)++ optimized_classifier(text="What does a pending cash withdrawal mean?")+ ```++ **Possible Output (from the last line):**+ ```text+ Prediction(+ reasoning='A pending cash withdrawal indicates that a request to withdraw cash has been initiated but has not yet been completed or processed. This status means that the transaction is still in progress and the funds have not yet been deducted from the account or made available to the user.',+ label='pending_cash_withdrawal'+ )+ ```++ An informal run similar to this on DSPy 2.5.29 raises GPT-4o-mini's score 66% to 87%.+++## Saving and loading optimizer output++After running a program through an optimizer, it's useful to also save it. At a later point, a program can be loaded from a file and used for inference. For this, the `load` and `save` methods can be used.++```python+optimized_program.save(YOUR_SAVE_PATH)+```++The resulting file is in plain-text JSON format. It contains all the parameters and steps in the source program. You can always read it and see what the optimizer generated. You can add `save_field_meta` to additionally save the list of fields with the keys, `name`, `field_type`, `description`, and `prefix` with: `optimized_program.save(YOUR_SAVE_PATH, save_field_meta=True).+++To load a program from a file, you can instantiate an object from that class and then call the load method on it.++```python+loaded_program = YOUR_PROGRAM_CLASS()+loaded_program.load(path=YOUR_SAVE_PATH)+```+diff --git a/docs/docs/learn/optimization/overview.md b/docs/docs/learn/optimization/overview.mdnew file mode 100644index 000000000..f8cb2b226--- /dev/null+++ b/docs/docs/learn/optimization/overview.md@@ -0,0 +1,13 @@+---+sidebar_position: 1+---+++# Optimization in DSPy++Once you have a system and a way to evaluate it, you can use DSPy optimizers to tune the prompts or weights in your program. Now it's useful to expand your data collection effort into building a training set and a held-out test set, in addition to the development set you've been using for exploration. For the training set (and its subset, validation set), you can often get substantial value out of 30 examples, but aim for at least 300 examples. Some optimizers accept a `trainset` only. Others ask for a `trainset` and a `valset`. For prompt optimizers, we suggest starting with a 20% split for training and 80% for validation, which is often the _opposite_ of what one does for DNNs.++After your first few optimization runs, you are either very happy with everything or you've made a lot of progress but you don't like something about the final program or the metric. At this point, go back to step 1 (Programming in DSPy) and revisit the major questions. Did you define your task well? Do you need to collect (or find online) more data for your problem? Do you want to update your metric? And do you want to use a more sophisticated optimizer? Do you need to consider advanced features like DSPy Assertions? Or, perhaps most importantly, do you want to add some more complexity or steps in your DSPy program itself? Do you want to use multiple optimizers in a sequence?++Iterative development is key. DSPy gives you the pieces to do that incrementally: iterating on your data, your program structure, your assertions, your metric, and your optimization steps. Optimizing complex LM programs is an entirely new paradigm that only exists in DSPy at the time of writing (update: there are now numerous DSPy extension frameworks, so this part is no longer true :-), so naturally the norms around what to do are still emerging. If you need help, we recently created a [Discord server](https://discord.gg/XCGy2WDCQB) for the community.+diff --git a/docs/docs/building-blocks/solving_your_task.md b/docs/docs/learn/optimization/solving_your_task.mdsimilarity index 96%rename from docs/docs/building-blocks/solving_your_task.mdrename to docs/docs/learn/optimization/solving_your_task.mdindex f4c59cd91..aad82b8e9 100644--- a/docs/docs/building-blocks/solving_your_task.md+++ b/docs/docs/learn/optimization/solving_your_task.md@@ -4,9 +4,7 @@ sidebar_position: 1# Using DSPy in 8 Steps-Using DSPy well for solving a new task is just doing good machine learning with LMs.--What this means is that it's an iterative process. You make some initial choices, which will be sub-optimal, and then you refine them incrementally.+Using DSPy well for solving a new task is just doing good machine learning with LMs. It's an iterative process: you make some initial choices, which will be sub-optimal, and then you refine them incrementally.As we discuss below, you will define your task and the metrics you want to maximize, and prepare a few example inputs — typically without labels (or only with labels for the final outputs, if your metric requires them). Then, you build your pipeline by selecting built-in layers [(`modules`)](/building-blocks/3-modules) to use, giving each layer a [`signature` (input/output spec)](/building-blocks/2-signatures), and then calling your modules freely in your Python code. Lastly, you use a DSPy [`optimizer`](/building-blocks/6-optimizers) to compile your code into high-quality instructions, automatic few-shot examples, or updated LM weights for your LM.@@ -17,9 +15,7 @@ You cannot use DSPy well if you haven't defined the problem you're trying to sol**Expected Input/Output Behavior:** Are you trying to build a chatbot over your data? A code assistant? A system for extracting information from papers? Or perhaps a translation system? Or a system for highlighting snippets from search results? Or a system to summarize information on a topic, with citations?-It's often useful to come up with just 3-4 examples of the inputs and outputs of your program (e.g., questions and their answers, or topics and their summaries).--If you need help thinking about your task, we recently created a [Discord server](https://discord.gg/XCGy2WDCQB) for the community.+It's often useful to come up with just 3-4 examples of the inputs and outputs of your program (e.g., questions and their answers, or topics and their summaries). If you need help thinking about your task, we recently created a [Discord server](https://discord.gg/XCGy2WDCQB) for the community.**Quality and Cost Specs:** You probably don't have infinite budget. Your final system can't be too expensive to run, and it should probably respond to users quickly enough.diff --git a/docs/docs/building-blocks/7-assertions.md b/docs/docs/learn/programming/7-assertions.mdsimilarity index 99%rename from docs/docs/building-blocks/7-assertions.mdrename to docs/docs/learn/programming/7-assertions.mdindex bfd207315..a2c46cbb1 100644--- a/docs/docs/building-blocks/7-assertions.md+++ b/docs/docs/learn/programming/7-assertions.md@@ -1,5 +1,8 @@# DSPy Assertions+!!! warning "This page is outdated and may not be fully accurate in DSPy 2.5"++## IntroductionLanguage models (LMs) have transformed how we interact with machine learning, offering vast capabilities in natural language understanding and generation. However, ensuring these models adhere to domain-specific constraints remains a challenge. Despite the growth of techniques like fine-tuning or “prompt engineering”, these approaches are extremely tedious and rely on heavy, manual hand-waving to guide the LMs in adhering to specific constraints. Even DSPy's modularity of programming prompting pipelines lacks mechanisms to effectively and automatically enforce these constraints.diff --git a/docs/docs/learn/programming/language_models.md b/docs/docs/learn/programming/language_models.mdnew file mode 100644index 000000000..93a4fd8aa--- /dev/null+++ b/docs/docs/learn/programming/language_models.md@@ -0,0 +1,167 @@+---+sidebar_position: 2+---++# Language Models++The first step in any DSPy code is to set up your language model. For example, you can configure OpenAI's GPT-4o-mini as your default LM as follows.++```python+# Authenticate via the `OPENAI_API_KEY` env variable or by passing an `api_key` argument below.+lm = dspy.LM('openai/gpt-4o-mini')+dspy.configure(lm=lm)+```++!!! info "Other LMs"++ === "Anthropic"+ You can authenticate by setting the ANTHROPIC_API_KEY env variable or passing `api_key` below.++ ```python linenums="1"+ import dspy+ lm = dspy.LM('anthropic/claude-3-opus-20240229', api_key='YOUR_ANTHROPIC_API_KEY')+ dspy.configure(lm=lm)+ ```++ === "Databricks"+ If you're on the Databricks platform, authentication is automatic via their SDK. If not, you can set the env variables `DATABRICKS_API_KEY` and `DATABRICKS_API_BASE`, or pass `api_key` and `api_base` below.++ ```python linenums="1"+ import dspy+ lm = dspy.LM('databricks/databricks-meta-llama-3-1-70b-instruct')+ dspy.configure(lm=lm)+ ```++ === "Local LMs on a GPU server"+ First, install [SGLang](https://sgl-project.github.io/start/install.html) and launch its server with your LM.++ ```bash+ > pip install "sglang[all]"+ > pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/++ > CUDA_VISIBLE_DEVICES=0 python -m sglang.launch_server --port 7501 --model-path meta-llama/Meta-Llama-3-8B-Instruct+ ```++ Then, connect to it from your DSPy code as an OpenAI-compatible endpoint.++ ```python linenums="1"+ lm = dspy.LM("openai/meta-llama/Meta-Llama-3-8B-Instruct",+ api_base="http://localhost:7501/v1", # ensure this points to your port+ api_key="", model_type='chat')+ dspy.configure(lm=lm)+ ```++ === "Local LMs on your laptop"+ First, install [Ollama](https://github.com/ollama/ollama) and launch its server with your LM.++ ```bash+ > curl -fsSL https://ollama.ai/install.sh | sh+ > ollama run llama3.2:1b+ ```++ Then, connect to it from your DSPy code.++ ```python linenums="1"+ import dspy+ lm = dspy.LM('ollama_chat/llama3.2', api_base='http://localhost:11434', api_key='')+ dspy.configure(lm=lm)+ ```++ === "Other providers"+ In DSPy, you can use any of the dozens of [LLM providers supported by LiteLLM](https://docs.litellm.ai/docs/providers). Simply follow their instructions for which `{PROVIDER}_API_KEY` to set and how to write pass the `{provider_name}/{model_name}` to the constructor.++ Some examples:++ - `anyscale/mistralai/Mistral-7B-Instruct-v0.1`, with `ANYSCALE_API_KEY`+ - `together_ai/togethercomputer/llama-2-70b-chat`, with `TOGETHERAI_API_KEY`+ - `sagemaker/<your-endpoint-name>`, with `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_REGION_NAME`+ - `azure/<your_deployment_name>`, with `AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION`, and the optional `AZURE_AD_TOKEN` and `AZURE_API_TYPE`+++ If your provider offers an OpenAI-compatible endpoint, just add an `openai/` prefix to your full model name.++ ```python linenums="1"+ import dspy+ lm = dspy.LM('openai/your-model-name', api_key='PROVIDER_API_KEY', api_base='YOUR_PROVIDER_URL')+ dspy.configure(lm=lm)+ ```++## Calling the LM directly.++It's easy to call the `lm` you configured above directly. This gives you a unified API and lets you benefit from utilities like automatic caching.++```python linenums="1"+lm("Say this is a test!", temperature=0.7) # => ['This is a test!']+lm(messages=[{"role": "user", "content": "Say this is a test!"}]) # => ['This is a test!']+```++## Using the LM with DSPy modules.++Idiomatic DSPy involves using _modules_, which we discuss in the next guide.++```python+# Define a module (ChainOfThought) and assign it a signature (return an answer, given a question).+qa = dspy.ChainOfThought('question -> answer')++# Run with the default LM configured with `dspy.configure` above.+response = qa(question="How many floors are in the castle David Gregory inherited?")+print(response.answer)+```+**Possible Output:**+```text+The castle David Gregory inherited has 7 floors.+```++## Using multiple LMs.++You can change the default LM globally with `dspy.configure` or change it inside a block of code with `dspy.context`.++!!! tip+ Using `dspy.configure` and `dspy.context` is thread-safe!+++```python+dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))+response = qa(question="How many floors are in the castle David Gregory inherited?")+print('GPT-4o-mini:', response.answer)++with dspy.context(lm=dspy.LM('openai/gpt-3.5-turbo')):+ response = qa(question="How many floors are in the castle David Gregory inherited?")+ print('GPT-3.5-turbo:', response.answer)+```+**Possible Output:**+```text+GPT-4o: The number of floors in the castle David Gregory inherited cannot be determined with the information provided.+GPT-3.5-turbo: The castle David Gregory inherited has 7 floors.+```++## Configuring LM generation.++For any LM, you can configure any of the following attributes at initialization or in each subsequent call.++```python+gpt_4o_mini = dspy.LM('openai/gpt-4o-mini', temperature=0.9, max_tokens=3000, stop=None, cache=False)+```++By default LMs in DSPy are cached. If you repeat the same call, you will get the same outputs. But you can turn off caching by setting `cache=False`.+++## Inspecting output and usage metadata.++Every LM object maintains the history of its interactions, including inputs, outputs, token usage (and $$$ cost), and metadata.++```python+len(lm.history) # e.g., 3 calls to the LM++lm.history[-1].keys() # access the last call to the LM, with all metadata+```++**Output:**+```text+dict_keys(['prompt', 'messages', 'kwargs', 'response', 'outputs', 'usage', 'cost'])+```++### Advanced: Building customer LMs and writing your own Adapters.++Though rarely needed, you can write custom LMs by inheriting from `dspy.BaseLM`. Another advanced layer in the DSPy ecosystem is that of _adapters_, which sit between DSPy signatures and LMs. A future version of this guide will discuss these advanced features, though you likely don't need them.+diff --git a/docs/docs/learn/programming/modules.md b/docs/docs/learn/programming/modules.mdnew file mode 100644index 000000000..dc75985b0--- /dev/null+++ b/docs/docs/learn/programming/modules.md@@ -0,0 +1,223 @@+---+sidebar_position: 3+---++# Modules++A **DSPy module** is a building block for programs that use LMs.++- Each built-in module abstracts a **prompting technique** (like chain of thought or ReAct). Crucially, they are generalized to handle any signature.++- A DSPy module has **learnable parameters** (i.e., the little pieces comprising the prompt and the LM weights) and can be invoked (called) to process inputs and return outputs.++- Multiple modules can be composed into bigger modules (programs). DSPy modules are inspired directly by NN modules in PyTorch, but applied to LM programs.+++## How do I use a built-in module, like `dspy.Predict` or `dspy.ChainOfThought`?++Let's start with the most fundamental module, `dspy.Predict`. Internally, all other DSPy modules are built using `dspy.Predict`. We'll assume you are already at least a little familiar with [DSPy signatures](/building-blocks/2-signatures), which are declarative specs for defining the behavior of any module we use in DSPy.++To use a module, we first **declare** it by giving it a signature. Then we **call** the module with the input arguments, and extract the output fields!++```python+sentence = "it's a charming and often affecting journey." # example from the SST-2 dataset.++# 1) Declare with a signature.+classify = dspy.Predict('sentence -> sentiment')++# 2) Call with input argument(s).+response = classify(sentence=sentence)++# 3) Access the output.+print(response.sentiment)+```+**Output:**+```text+Positive+```++When we declare a module, we can pass configuration keys to it.++Below, we'll pass `n=5` to request five completions. We can also pass `temperature` or `max_len`, etc.++Let's use `dspy.ChainOfThought`. In many cases, simply swapping `dspy.ChainOfThought` in place of `dspy.Predict` improves quality.++```python+question = "What's something great about the ColBERT retrieval model?"++# 1) Declare with a signature, and pass some config.+classify = dspy.ChainOfThought('question -> answer', n=5)++# 2) Call with input argument.+response = classify(question=question)++# 3) Access the outputs.+response.completions.answer+```+**Output:**+```text+['One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.',+ 'Its ability to efficiently retrieve relevant information from large document collections.',+ 'One great thing about the ColBERT retrieval model is its superior performance compared to other models and its efficient use of pre-trained language models.',+ 'One great thing about the ColBERT retrieval model is its superior efficiency and accuracy compared to other models.',+ 'One great thing about the ColBERT retrieval model is its ability to incorporate user feedback and support complex queries.']+```++Let's discuss the output object here. The `dspy.ChainOfThought` module will generally inject a `reasoning` before the output field(s) of your signature.++Let's inspect the (first) reasoning and answer!++```python+print(f"Reasoning: {response.reasoning}")+print(f"Answer: {response.answer}")+```+**Output:**+```text+Rationale: produce the answer. We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.+Answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.+```++This is accessible whether we request one or many completions.++We can also access the different completions as a list of `Prediction`s or as several lists, one for each field.++```python+response.completions[3].reasoning == response.completions.reasoning[3]+```+**Output:**+```text+True+```+++## What other DSPy modules are there? How can I use them?++The others are very similar. They mainly change the internal behavior with which your signature is implemented!++1. **`dspy.Predict`**: Basic predictor. Does not modify the signature. Handles the key forms of learning (i.e., storing the instructions and demonstrations and updates to the LM).++2. **`dspy.ChainOfThought`**: Teaches the LM to think step-by-step before committing to the signature's response.++3. **`dspy.ProgramOfThought`**: Teaches the LM to output code, whose execution results will dictate the response.++4. **`dspy.ReAct`**: An agent that can use tools to implement the given signature.++5. **`dspy.MultiChainComparison`**: Can compare multiple outputs from `ChainOfThought` to produce a final prediction.+++We also have some function-style modules:++6. **`dspy.majority`**: Can do basic voting to return the most popular response from a set of predictions.+++!!! info "A few examples of DSPy modules on simple tasks."+ Try the examples below after configuring your `lm`. Adjust the fields to explore what tasks your LM can do well out of the box.++ === "Math"++ ```python linenums="1"+ math = dspy.ChainOfThought("question -> answer: float")+ math(question="Two dice are tossed. What is the probability that the sum equals two?")+ ```++ **Possible Output:**+ ```text+ Prediction(+ reasoning='When two dice are tossed, each die has 6 faces, resulting in a total of 6 x 6 = 36 possible outcomes. The sum of the numbers on the two dice equals two only when both dice show a 1. This is just one specific outcome: (1, 1). Therefore, there is only 1 favorable outcome. The probability of the sum being two is the number of favorable outcomes divided by the total number of possible outcomes, which is 1/36.',+ answer=0.0277776+ )+ ```++ === "Retrieval-Augmented Generation"++ ```python linenums="1"+ def search(query: str) -> list[str]:+ """Retrieves abstracts from Wikipedia."""+ results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)+ return [x['text'] for x in results]++ rag = dspy.ChainOfThought('context, question -> response')++ question = "What's the name of the castle that David Gregory inherited?"+ rag(context=search(question), question=question)+ ```++ **Possible Output:**+ ```text+ Prediction(+ reasoning='The context provides information about David Gregory, a Scottish physician and inventor. It specifically mentions that he inherited Kinnairdy Castle in 1664. This detail directly answers the question about the name of the castle that David Gregory inherited.',+ response='Kinnairdy Castle'+ )+ ```++ === "Classification"++ ```python linenums="1"+ from typing import Literal++ class Classify(dspy.Signature):+ """Classify sentiment of a given sentence."""++ sentence: str = dspy.InputField()+ sentiment: Literal['positive', 'negative', 'neutral'] = dspy.OutputField()+ confidence: float = dspy.OutputField()++ classify = dspy.Predict(Classify)+ classify(sentence="This book was super fun to read, though not the last chapter.")+ ```++ **Possible Output:**++ ```text+ Prediction(+ sentiment='positive',+ confidence=0.75+ )+ ```++ === "Information Extraction"++ ```python linenums="1"+ text = "Apple Inc. announced its latest iPhone 14 today. The CEO, Tim Cook, highlighted its new features in a press release."++ module = dspy.Predict("text -> title, headings: list[str], entities_and_metadata: list[dict[str, str]]")+ response = module(text=text)++ print(response.title)+ print(response.headings)+ print(response.entities_and_metadata)+ ```++ **Possible Output:**+ ```text+ Apple Unveils iPhone 14+ ['Introduction', 'Key Features', "CEO's Statement"]+ [{'entity': 'Apple Inc.', 'type': 'Organization'}, {'entity': 'iPhone 14', 'type': 'Product'}, {'entity': 'Tim Cook', 'type': 'Person'}]+ ```++ === "Agents"++ ```python linenums="1"+ def evaluate_math(expression: str) -> float:+ return dspy.PythonInterpreter({}).execute(expression)++ def search_wikipedia(query: str) -> str:+ results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)+ return [x['text'] for x in results]++ react = dspy.ReAct("question -> answer: float", tools=[evaluate_math, search_wikipedia])++ pred = react(question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?")+ print(pred.answer)+ ```++ **Possible Output:**++ ```text+ 5761.328+ ```+++## How do I compose multiple modules into a bigger program?++DSPy is just Python code that uses modules in any control flow you like, with a little magic internally at `compile` time to trace your LM calls. What this means is that, you can just call the modules freely. No weird abstractions for chaining calls. This is basically PyTorch's design approach for define-by-run / dynamic computation graphs. Refer to the intro tutorials for examples.diff --git a/docs/docs/learn/programming/overview.md b/docs/docs/learn/programming/overview.mdnew file mode 100644index 000000000..86c97037a--- /dev/null+++ b/docs/docs/learn/programming/overview.md@@ -0,0 +1,18 @@+---+sidebar_position: 1+---++# Programming in DSPy++DSPy is fundamentally a bet on _writing code instead of strings_: your control flow is often the most important part of your AI system. To design this, the first step is to **define your task**. What is the expected input/output behavior of your system? For example, are you trying to build a chatbot over your data or perhaps a code assistant? Or maybe a system for translation, for highlighting snippets from search results, or for generating reports with citations?++The next step is to **define your initial pipeline**. Can your DSPy program just be a simple chain-of-thought module? Do you need the LM to use retrieval or perhaps other tools, like a calculator or a calendar API? Is there a typical workflow for solving your problem in multiple well-scoped steps, or do you want more open-ended tool use with agents for your task? Think about this space but always start simple. Almost every task should probably start with just a single `dspy.ChainOfThought` module then add complexity incrementally based on observations.++As you define your task and your pipeline, **craft and try a handful of examples** of the inputs to your program. Consider using a powerful LM at this point, or a couple of different LMs, just to understand what's possible. Record the interesting (both easy and hard) examples you try. This will be useful when you are doing evaluation and optimization later.+++??? "Beyond encouraging good design patterns, how does DSPy help here?"++ Conventional prompts couple your fundamental system architecture with incidental choices not portable to new LMs, objectives, or pipelines. A conventional prompt asks the LM to take some inputs and produce some outputs of certain types (a _signature_), formats the inputs in certain ways and requests outputs in a form it can parse accurately (an _adapter_), asks the LM to apply certain strategies like "thinking step by step" or using tools (a _module_'s logic), and relies on substantial trial-and-error to discover the right way to ask each LM to do this (a form of manual _optimization_).++ DSPy separates these concerns and automates the lower-level ones until you need to consider them. This allow you to write much shorter code, with much higher portability. For example, if you write a program using DSPy modules, you can swap the LM or its adapter without changing the rest of your logic. Or you can exchange one _module_, like `dspy.ChainOfThought`, with another, like `dspy.ProgramOfThought`, without modifying your signatures. When you're ready to use optimizers, the same program can have its prompts optimized or its LM weights fine-tuned.diff --git a/docs/docs/building-blocks/2-signatures.md b/docs/docs/learn/programming/signatures.mdsimilarity index 85%rename from docs/docs/building-blocks/2-signatures.mdrename to docs/docs/learn/programming/signatures.mdindex 43302ebad..af400b11a 100644--- a/docs/docs/building-blocks/2-signatures.md+++ b/docs/docs/learn/programming/signatures.md@@ -8,19 +8,11 @@ When we assign tasks to LMs in DSPy, we specify the behavior we need as a Signat**A signature is a declarative specification of input/output behavior of a DSPy module.** Signatures allow you to tell the LM _what_ it needs to do, rather than specify _how_ we should ask the LM to do it.-You're probably familiar with function signatures, which specify the input and output arguments and their types. DSPy signatures are similar, but the differences are that:--- While typical function signatures just _describe_ things, DSPy Signatures _declare and initialize the behavior_ of modules.--- The field names matter in DSPy Signatures. You express semantic roles in plain English: a `question` is different from an `answer`, a `sql_query` is different from `python_code`.+You're probably familiar with function signatures, which specify the input and output arguments and their types. DSPy signatures are similar, but with a couple of differences. While typical function signatures just _describe_ things, DSPy Signatures _declare and initialize the behavior_ of modules. Moreover, the field names matter in DSPy Signatures. You express semantic roles in plain English: a `question` is different from an `answer`, a `sql_query` is different from `python_code`.## Why should I use a DSPy Signature?-**tl;dr** For modular and clean code, in which LM calls can be optimized into high-quality prompts (or automatic finetunes).--**Long Answer:** Most people coerce LMs to do tasks by hacking long, brittle prompts. Or by collecting/generating data for fine-tuning.--Writing signatures is far more modular, adaptive, and reproducible than hacking at prompts or finetunes. The DSPy compiler will figure out how to build a highly-optimized prompt for your LM (or finetune your small LM) for your signature, on your data, and within your pipeline. In many cases, we found that compiling leads to better prompts than humans write. Not because DSPy optimizers are more creative than humans, but simply because they can try more things and tune the metrics directly.+For modular and clean code, in which LM calls can be optimized into high-quality prompts (or automatic finetunes). Most people coerce LMs to do tasks by hacking long, brittle prompts. Or by collecting/generating data for fine-tuning. Writing signatures is far more modular, adaptive, and reproducible than hacking at prompts or finetunes. The DSPy compiler will figure out how to build a highly-optimized prompt for your LM (or finetune your small LM) for your signature, on your data, and within your pipeline. In many cases, we found that compiling leads to better prompts than humans write. Not because DSPy optimizers are more creative than humans, but simply because they can try more things and tune the metrics directly.## **Inline** DSPy Signaturesdiff --git a/docs/mkdocs.yml b/docs/mkdocs.ymlindex 597800615..bd534f1b7 100644--- a/docs/mkdocs.yml+++ b/docs/mkdocs.yml@@ -10,51 +10,38 @@ docs_dir: 'docs/'nav:- Home: index.md- Learn DSPy:- - Overview: building-blocks/solving_your_task.md- - Language Models: building-blocks/1-language_models.md- - Signatures: building-blocks/2-signatures.md- - Data Handling: building-blocks/4-data.md- - Modules:- - Overview: building-blocks/3-modules.md- - Predict: deep-dive/modules/predict.md- - ChainOfThought: deep-dive/modules/chain-of-thought.md- - ChainOfThoughtWithHint: deep-dive/modules/chain-of-thought-with-hint.md- - ReAct: deep-dive/modules/react.md- - MultiChainComparison: deep-dive/modules/multi-chain-comparison.md- - ProgramOfThought: deep-dive/modules/program-of-thought.md- - Retrieve: deep-dive/modules/retrieve.md- - Modules Guide: deep-dive/modules/guide.md- - Metrics:- - Metrics: building-blocks/5-metrics.md- - Assertions: building-blocks/7-assertions.md- - Assertions II: deep-dive/assertions.md- - Optimizers:- - Overview: building-blocks/6-optimizers.md- - LabeledFewShot: deep-dive/optimizers/LabeledFewShot.md- - BootstrapFewShot: deep-dive/optimizers/bootstrap-fewshot.md- - BootstrapRS: deep-dive/optimizers/bfrs.md- - COPRO: deep-dive/optimizers/copro.md- - MIPROv2: deep-dive/optimizers/miprov2.md- - Ensemble: deep-dive/optimizers/Ensemble.md- - BootstrapFinetune: deep-dive/optimizers/BootstrapFinetune.md- - Retrieval Clients:- - Azure: deep-dive/retrieval_models_clients/Azure.md- - ChromadbRM: deep-dive/retrieval_models_clients/ChromadbRM.md- - ClarifaiRM: deep-dive/retrieval_models_clients/ClarifaiRM.md- - ColBERTv2: deep-dive/retrieval_models_clients/ColBERTv2.md- - Custom RM Client: deep-dive/retrieval_models_clients/custom-rm-client.md- - DatabricksRM: deep-dive/retrieval_models_clients/DatabricksRM.md- - FaissRM: deep-dive/retrieval_models_clients/FaissRM.md- - LancedbRM: deep-dive/retrieval_models_clients/LancedbRM.md- - MilvusRM: deep-dive/retrieval_models_clients/MilvusRM.md- - MyScaleRM: deep-dive/retrieval_models_clients/MyScaleRM.md- - Neo4jRM: deep-dive/retrieval_models_clients/Neo4jRM.md- - QdrantRM: deep-dive/retrieval_models_clients/QdrantRM.md- - RAGatouilleRM: deep-dive/retrieval_models_clients/RAGatouilleRM.md- - SnowflakeRM: deep-dive/retrieval_models_clients/SnowflakeRM.md- - WatsonDiscovery: deep-dive/retrieval_models_clients/WatsonDiscovery.md- - WeaviateRM: deep-dive/retrieval_models_clients/WeaviateRM.md- - YouRM: deep-dive/retrieval_models_clients/YouRM.md+ - Learning DSPy: learn/index.md+ - DSPy Programming:+ - Programming Overview: learn/programming/overview.md+ - Language Models: learn/programming/language_models.md+ - Signatures: learn/programming/signatures.md+ - Modules: learn/programming/modules.md+ - DSPy Evaluation:+ - Evaluation Overview: learn/evaluation/overview.md+ - Data Handling: learn/evaluation/data.md+ - Metrics: learn/evaluation/metrics.md+ - DSPy Optimization:+ - Optimization Overview: learn/optimization/overview.md+ - Optimizers: learn/optimization/optimizers.md+ - Other References:+ - Retrieval Clients:+ - Azure: deep-dive/retrieval_models_clients/Azure.md+ - ChromadbRM: deep-dive/retrieval_models_clients/ChromadbRM.md+ - ClarifaiRM: deep-dive/retrieval_models_clients/ClarifaiRM.md+ - ColBERTv2: deep-dive/retrieval_models_clients/ColBERTv2.md+ - Custom RM Client: deep-dive/retrieval_models_clients/custom-rm-client.md+ - DatabricksRM: deep-dive/retrieval_models_clients/DatabricksRM.md+ - FaissRM: deep-dive/retrieval_models_clients/FaissRM.md+ - LancedbRM: deep-dive/retrieval_models_clients/LancedbRM.md+ - MilvusRM: deep-dive/retrieval_models_clients/MilvusRM.md+ - MyScaleRM: deep-dive/retrieval_models_clients/MyScaleRM.md+ - Neo4jRM: deep-dive/retrieval_models_clients/Neo4jRM.md+ - QdrantRM: deep-dive/retrieval_models_clients/QdrantRM.md+ - RAGatouilleRM: deep-dive/retrieval_models_clients/RAGatouilleRM.md+ - SnowflakeRM: deep-dive/retrieval_models_clients/SnowflakeRM.md+ - WatsonDiscovery: deep-dive/retrieval_models_clients/WatsonDiscovery.md+ - WeaviateRM: deep-dive/retrieval_models_clients/WeaviateRM.md+ - YouRM: deep-dive/retrieval_models_clients/YouRM.md- Tutorials:- Simple RAG: tutorials/rag/index.ipynb- Community: