Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Update bfrs docs and docs order

Herumb Shandilyace1c909283a5
diff --git a/docs/docs/deep-dive/optimizers/BootstrapFinetune.md b/docs/docs/deep-dive/optimizers/BootstrapFinetune.md
index 86540aa46..941848523 100644
--- a/docs/docs/deep-dive/optimizers/BootstrapFinetune.md
+++ b/docs/docs/deep-dive/optimizers/BootstrapFinetune.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 5
+sidebar_position: 4
---
# teleprompt.BootstrapFinetune
diff --git a/docs/docs/deep-dive/optimizers/Ensemble.md b/docs/docs/deep-dive/optimizers/Ensemble.md
index 42dbc0dde..eb9ddb61b 100644
--- a/docs/docs/deep-dive/optimizers/Ensemble.md
+++ b/docs/docs/deep-dive/optimizers/Ensemble.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 3
+sidebar_position: 4
---
# teleprompt.Ensemble
diff --git a/docs/docs/deep-dive/optimizers/LabeledFewShot.md b/docs/docs/deep-dive/optimizers/LabeledFewShot.md
index d58f688aa..c815a8d18 100644
--- a/docs/docs/deep-dive/optimizers/LabeledFewShot.md
+++ b/docs/docs/deep-dive/optimizers/LabeledFewShot.md
@@ -1,8 +1,8 @@
---
-sidebar_position: 1
+sidebar_position: 2
---
-# teleprompt.LabeledFewShot
+# LabeledFewShot
### Constructor
diff --git a/docs/docs/deep-dive/optimizers/bfrs.md b/docs/docs/deep-dive/optimizers/bfrs.md
new file mode 100644
index 000000000..f57613248
--- /dev/null
+++ b/docs/docs/deep-dive/optimizers/bfrs.md
@@ -0,0 +1,54 @@
+---
+sidebar_position: 3
+---
+
+# BootstrapFewShotWithRandomSearch
+
+`BootstrapFewShotWithRandomSearch` is an teleprompter that extends `BootstrapFewShot` by incorporating a random search strategy to optimize the selection of few-shot examples. This teleprompter is useful when you want to explore a variety of example combinations to find the best examples for your model.
+
+## Usage
+
+In terms of API `BootstrapFewShotWithRandomSearch` teleprompter is quite similar to `BootstrapFewShot` with few additional parameters needed. More precisely the parameters are:
+
+- `metric` (callable): The metric function used to evaluate examples during bootstrapping and final evaluation.
+- `teacher_settings` (dict, optional): Settings for the teacher predictor. Defaults to an empty dictionary.
+- `max_bootstrapped_demos` (int, optional): Maximum number of bootstrapped demonstrations per predictor. Defaults to 4.
+- `max_labeled_demos` (int, optional): Maximum number of labeled demonstrations per predictor. Defaults to 16.
+- `max_rounds` (int, optional): Maximum number of bootstrapping rounds. Defaults to 1.
+- `num_candidate_programs` (int): Number of candidate programs to generate during random search. Defaults to 16.
+- `num_threads` (int): Number of threads used for evaluation during random search. Defaults to 6.
+- `max_errors` (int): Maximum errors permitted during evaluation. Halts run with the latest error message. Defaults to 10.
+- `stop_at_score` (float, optional): Score threshold for random search to stop early. Defaults to None.
+- `metric_threshold` (float, optional): Score threshold for the metric to determine a successful example. Defaults to None.
+
+## Working Example
+
+Let's take the example of optimizing a simple CoT pipeline for GSM8k dataset, we'll take the example in [BootstrapFewShot](/docs/deep-dive/optimizers/bootstrap-fewshot.mdx) as our running example for optimizers. We're gonna assume our data and pipeline is same as the on in `BootstrapFewShot` article. So let's start by intializing the optimizer:
+
+```python
+from dspy.teleprompt import BootstrapFewShotWithRandomSearch
+
+teleprompter = BootstrapFewShotWithRandomSearch(
+ metric=gsm8k_metric,
+ max_bootstrapped_demos=8,
+ max_labeled_demos=8,
+ num_threads=10,
+ num_candidate_programs=10
+)
+```
+
+Now that we have initialized our teleprompter, let's compile our CoT module with the call to `compile` standard to all optimizers:
+
+```python
+cot_compiled = teleprompter.compile(CoT(), trainset=trainset, valset=devset)
+```
+
+Once the training is done you'll have a more optimized module that you can save or load again for use anytime:
+
+```python
+cot_compiled.save('turbo_gsm8k.json')
+
+# Loading:
+# cot = CoT()
+# cot.load('turbo_gsm8k.json')
+```
\ No newline at end of file
diff --git a/docs/docs/deep-dive/optimizers/bfrs.mdx b/docs/docs/deep-dive/optimizers/bfrs.mdx
deleted file mode 100644
index 214d588ac..000000000
--- a/docs/docs/deep-dive/optimizers/bfrs.mdx
+++ /dev/null
@@ -1,28 +0,0 @@
----
-sidebar_position: 4
----
-
-# teleprompt.BootstrapFewShotWithRandomSearch
-
-### Constructor
-
-The constructor initializes the `BootstrapFewShotWithRandomSearch` class and sets up its attributes. It inherits from the `BootstrapFewShot` class and introduces additional attributes for the random search process.
-
-### Method
-
-Refer to [teleprompt.BootstrapFewShot](https://dspy-docs.vercel.app/docs/deep-dive/teleprompter/bootstrap-fewshot) documentation.
-
-## Example
-
-```python
-#Assume defined trainset
-#Assume defined RAG class
-...
-
-#Define teleprompter and include teacher
-teacher = dspy.OpenAI(model='gpt-3.5-turbo', api_key = openai.api_key, api_provider = "openai", model_type = "chat")
-teleprompter = BootstrapFewShotWithRandomSearch(teacher_settings=dict({'lm': teacher}))
-
-# Compile!
-compiled_rag = teleprompter.compile(student=RAG(), trainset=trainset)
-```
\ No newline at end of file
diff --git a/docs/docs/deep-dive/optimizers/bootstrap-fewshot.mdx b/docs/docs/deep-dive/optimizers/bootstrap-fewshot.mdx
index 4d15d6709..9902d651f 100644
--- a/docs/docs/deep-dive/optimizers/bootstrap-fewshot.mdx
+++ b/docs/docs/deep-dive/optimizers/bootstrap-fewshot.mdx
@@ -15,18 +15,25 @@ We'll be making a basic answer generation pipeline over GSM8K dataset that we sa
```python
import dspy
-turbo = dspy.OpenAI(model='gpt-3.5-turbo', max_tokens=250)
-dspy.settings.configure(lm=turbo)
+turbo = dspy.LM(model='openai/gpt-3.5-turbo', max_tokens=250)
+dspy.configure(lm=turbo)
```
Now that we have the LM client setup it's time to import the train-dev split in `GSM8k` class that DSPy provides us:
```python
-from dspy.datasets.gsm8k import GSM8K, gsm8k_metric
+import random
+from dspy.datasets import DataLoader
-gms8k = GSM8K()
+dl = DataLoader()
-trainset, devset = gms8k.train, gms8k.dev
+gsm8k = dl.from_huggingface(
+ "gsm8k",
+ "main",
+ input_keys = ("question",),
+)
+
+trainset, devset = gsm8k['train'], random.sample(gsm8k['test'], 50)
```
We'll now define a basic QA inline signature i.e. `question->answer` and pass it to `ChainOfThought` module, that applies necessary addition for CoT style prompting to the Signature.
@@ -64,9 +71,9 @@ Now we have the baseline pipeline ready to use, so let's try using the `Bootstra
Let's start by importing and initializing our teleprompter, for the metric we'll be using the same `gsm8k_metric` imported and used above:
```python
-from dspy.teleprompt import BootstrapFewShotWithRandomSearch
+from dspy.teleprompt import BootstrapFewShot
-teleprompter = BootstrapFewShotWithRandomSearch(
+teleprompter = BootstrapFewShot(
metric=gsm8k_metric,
max_bootstrapped_demos=8,
max_labeled_demos=8,
diff --git a/docs/docs/deep-dive/optimizers/copro.mdx b/docs/docs/deep-dive/optimizers/copro.mdx
index 9ed818db7..22cf2dbca 100644
--- a/docs/docs/deep-dive/optimizers/copro.mdx
+++ b/docs/docs/deep-dive/optimizers/copro.mdx
@@ -1,5 +1,5 @@
---
-sidebar_position: 2
+sidebar_position: 5
---
import AuthorDetails from '@site/src/components/AuthorDetails';
diff --git a/docs/docs/deep-dive/optimizers/miprov2.md b/docs/docs/deep-dive/optimizers/miprov2.md
index 1ef388ea1..554901c7f 100644
--- a/docs/docs/deep-dive/optimizers/miprov2.md
+++ b/docs/docs/deep-dive/optimizers/miprov2.md
@@ -1,3 +1,7 @@
+---
+sidebar_position: 6
+---
+
# `MIPROv2` Optimizer
## Table of Contents
diff --git a/docs/docusaurus.config.ts b/docs/docusaurus.config.ts
index ca922e56d..ce1d9086c 100644
--- a/docs/docusaurus.config.ts
+++ b/docs/docusaurus.config.ts
@@ -55,17 +55,17 @@ const config: Config = {
// Plugins configuration
plugins: [
// Additional plugin for API documentation
- [
- '@docusaurus/plugin-content-docs',
- {
- id: 'api', // Unique identifier for the API docs
- path: './api', // Path to the API documentation markdown files
- routeBasePath: '/api/', // URL route for the API documentation section
- sidebarPath: require.resolve('./sidebars.ts'), // Path to the API sidebar configuration
- // URL for the "edit this page" feature for the API docs
- // editUrl: 'https://github.com/stanfordnlp/dspy/tree/main/api',
- },
- ],
+ //[
+ // '@docusaurus/plugin-content-docs',
+ // {
+ // id: 'api', // Unique identifier for the API docs
+ // path: './api', // Path to the API documentation markdown files
+ // routeBasePath: '/api/', // URL route for the API documentation section
+ // sidebarPath: require.resolve('./sidebars.ts'), // Path to the API sidebar configuration
+ // // URL for the "edit this page" feature for the API docs
+ // // editUrl: 'https://github.com/stanfordnlp/dspy/tree/main/api',
+ // },
+ //],
],
// Theme configuration
themeConfig: {