Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Build fix and cheatsheet update

Herumb Shandilya10111203499a
diff --git a/docs/building-blocks/1-language_models.md b/docs/building-blocks/1-language_models.md
index 87d772628..1457f6cac 100644
--- a/docs/building-blocks/1-language_models.md
+++ b/docs/building-blocks/1-language_models.md
@@ -153,13 +153,13 @@ tgi_llama2 = dspy.HFClientTGI(model="meta-llama/Llama-2-7b-hf", port=8080, url="
vllm_llama2 = dspy.HFClientVLLM(model="meta-llama/Llama-2-7b-hf", port=8080, url="http://localhost")
```
-3. `dspy.HFModel` (experimental) [Tutorial: How do I initialize models using HFModel](/api/hosting_language_models_locally/HFModel.md)
+3. `dspy.HFModel` (experimental) [Tutorial: How do I initialize models using HFModel](/api/hosting_language_models_locally/HFModel)
```python
llama = dspy.HFModel(model = 'meta-llama/Llama-2-7b-hf')
```
-4. `dspy.Ollama` (experimental) for open source models through [Ollama](https://ollama.com). [Tutorial: How do I install and use Ollama on a local computer?](/api/hosting_language_models_locally/Ollama.md)\n",
+4. `dspy.Ollama` (experimental) for open source models through [Ollama](https://ollama.com). [Tutorial: How do I install and use Ollama on a local computer?](/api/hosting_language_models_locally/Ollama)\n",
```python
mistral_ollama = dspy.OllamaLocal(model='mistral')
diff --git a/docs/building-blocks/solving_your_task.md b/docs/building-blocks/solving_your_task.md
index 332500275..2d90567bf 100644
--- a/docs/building-blocks/solving_your_task.md
+++ b/docs/building-blocks/solving_your_task.md
@@ -8,7 +8,7 @@ Using DSPy well for solving a new task is just doing good machine learning with
What this means is that it's an iterative process. You make some initial choices, which will be sub-optimal, and then you refine them incrementally.
-As we discuss below, you will define your task and the metrics you want to maximize, and prepare a few example inputs — typically without labels (or only with labels for the final outputs, if your metric requires them). Then, you build your pipeline by selecting built-in layers [(`modules`)](/docs/building-blocks/3-modules.md) to use, giving each layer a [`signature` (input/output spec)](/docs/building-blocks/2-signatures.md), and then calling your modules freely in your Python code. Lastly, you use a DSPy [`optimizer`](/docs/building-blocks/6-optimizers.md) to compile your code into high-quality instructions, automatic few-shot examples, or updated LM weights for your LM.
+As we discuss below, you will define your task and the metrics you want to maximize, and prepare a few example inputs — typically without labels (or only with labels for the final outputs, if your metric requires them). Then, you build your pipeline by selecting built-in layers [(`modules`)](/docs/building-blocks/modules) to use, giving each layer a [`signature` (input/output spec)](/docs/building-blocks/signatures), and then calling your modules freely in your Python code. Lastly, you use a DSPy [`optimizer`](/docs/building-blocks/optimizers) to compile your code into high-quality instructions, automatic few-shot examples, or updated LM weights for your LM.
## 1) Define your task.
@@ -31,7 +31,7 @@ What should your DSPy program do? Can it just be a simple chain-of-thought step?
Is there a typical workflow for solving your problem in multiple well-defined steps? Or do you want a fully open-ended LM (or open-ended tool use with agents) for your task?
-Think about this space but always start simple. Almost every task should probably start with just a single [dspy.ChainofThought](/api/modules/ChainOfThought.md) module, and then add complexity incrementally as you go.
+Think about this space but always start simple. Almost every task should probably start with just a single [dspy.ChainofThought](/api/modules/ChainOfThought) module, and then add complexity incrementally as you go.
Then write your (initial) DSPy program. Again: start simple, and let the next few steps guide any complexity you will add.
@@ -39,7 +39,7 @@ Then write your (initial) DSPy program. Again: start simple, and let the next fe
By this point, you probably have a few examples of the task you're trying to solve.
-Run them through your pipeline. Consider using a large and powerful LM at this point, or a couple of different LMs, just to understand what's possible. (DSPy will make swapping these LMs pretty easy - [LM Guide](/docs/building-blocks/1-language_models.md).)
+Run them through your pipeline. Consider using a large and powerful LM at this point, or a couple of different LMs, just to understand what's possible. (DSPy will make swapping these LMs pretty easy - [LM Guide](/docs/building-blocks/language_models).)
At this point, you're still using your pipeline zero-shot, so it will be far from perfect. DSPy will help you optimize the instructions, few-shot examples, and even weights of your LM calls below, but understanding where things go wrong in zero-shot usage will go a long way.
@@ -47,7 +47,7 @@ Record the interesting (both easy and hard) examples you try: even if you don't
## 4) Define your data.
-Now it's time to more formally declare your training and validation data for DSPy evaluation and optimization - [Data Guide](/docs/building-blocks/4-data.md).
+Now it's time to more formally declare your training and validation data for DSPy evaluation and optimization - [Data Guide](/docs/building-blocks/data).
You can use DSPy optimizers usefully with as few as 10 examples, but having 50-100 examples (or even better, 300-500 examples) goes a long way.
@@ -62,7 +62,7 @@ If there's data whose licenses are permissive enough, we suggest you use them. O
What makes outputs from your system good or bad? Invest in defining metrics and improving them over time incrementally. It's really hard to consistently improve what you aren't able to define.
-A metric is just a function that will take examples from your data and take the output of your system, and return a score that quantifies how good the output is - [Metric Guide](/docs/building-blocks/5-metrics.md).
+A metric is just a function that will take examples from your data and take the output of your system, and return a score that quantifies how good the output is - [Metric Guide](/docs/building-blocks/metrics).
For simple tasks, this could be just "accuracy" or "exact match" or "F1 score". This may be the case for simple classification or short-form QA tasks.
@@ -79,17 +79,17 @@ Look at the outputs and the metric scores. This will probably allow you to spot
## 7) Compile with a DSPy optimizer.
-Given some data and a metric, we can now optimize the program you built - [Optimizer Guide](/docs/building-blocks/6-optimizers.md).
+Given some data and a metric, we can now optimize the program you built - [Optimizer Guide](/docs/building-blocks/optimizers).
DSPy includes many optimizers that do different things. Remember: DSPy optimizers will create examples of each step, craft instructions, and/or update LM weights. In general, you don't need to have labels for your pipeline steps, but your data examples need to have input values and whatever labels your metric requires (e.g., no labels if your metric is reference-free, but final output labels otherwise in most cases).
Here's the general guidance on getting started:
-* If you have very little data, e.g. 10 examples of your task, use [`BootstrapFewShot`](/docs/deep-dive/teleprompter/bootstrap-fewshot.mdx)
+* If you have very little data, e.g. 10 examples of your task, use [`BootstrapFewShot`](/docs/deep-dive/teleprompter/bootstrap-fewshot)
-* If you have slightly more data, e.g. 50 examples of your task, use [`BootstrapFewShotWithRandomSearch`](/docs/deep-dive/teleprompter/bootstrap-fewshot.mdx).
+* If you have slightly more data, e.g. 50 examples of your task, use [`BootstrapFewShotWithRandomSearch`](/docs/deep-dive/teleprompter/bootstrap-fewshot).
-* If you have more data than that, e.g. 300 examples or more, use [`BayesianSignatureOptimizer`](/docs/deep-dive/teleprompter/signature-optimizer.mdx).
+* If you have more data than that, e.g. 300 examples or more, use [`BayesianSignatureOptimizer`](/docs/deep-dive/teleprompter/signature-optimizer).
* If you have been able to use one of these with a large LM (e.g., 7B parameters or above) and need a very efficient program, compile that down to a small LM with `BootstrapFinetune`.
@@ -97,7 +97,7 @@ Here's the general guidance on getting started:
At this point, you are either very happy with everything (we've seen quite a few people get it right on first try with DSPy) or, more likely, you've made a lot of progress but you don't like something about the final program or the metric.
-At this point, go back to step 1 and revisit the major questions. Did you define your task well? Do you need to collect (or find online) more data for your problem? Do you want to update your metric? And do you want to use a more sophisticated optimizer? Do you need to consider advanced features like [DSPy Assertions](/docs/deep-dive/modules/assertions.mdx)? Or, perhaps most importantly, do you want to add some more complexity or steps in your DSPy program itself? Do you want to use multiple optimizers in a sequence?
+At this point, go back to step 1 and revisit the major questions. Did you define your task well? Do you need to collect (or find online) more data for your problem? Do you want to update your metric? And do you want to use a more sophisticated optimizer? Do you need to consider advanced features like [DSPy Assertions](/docs/deep-dive/modules/assertions)? Or, perhaps most importantly, do you want to add some more complexity or steps in your DSPy program itself? Do you want to use multiple optimizers in a sequence?
Iterative development is key. DSPy gives you the pieces to do that incrementally: iterating on your data, your program structure, your assertions, your metric, and your optimization steps.
diff --git a/docs/cheatsheet.md b/docs/cheatsheet.md
index acecc04b7..b391c0b75 100644
--- a/docs/cheatsheet.md
+++ b/docs/cheatsheet.md
@@ -7,9 +7,118 @@ sidebar_position: 999
This page will contain snippets for frequent usage patterns.
-## DSPy Datasets.
+## DSPy DataLoaders
-TODO
+Import and initializing a DataLoader Object:
+
+```python
+import dspy
+from dspy.datasets import DataLoader
+
+dl = DataLoader()
+```
+
+### Loading from HuggingFace Datasets
+
+```python
+code_alpaca = dl.from_huggingface("HuggingFaceH4/CodeAlpaca_20K")
+```
+
+You can access the dataset of the splits by calling key of the corresponding split:
+
+```python
+train_dataset = code_alpaca['train']
+test_dataset = code_alpaca['test']
+```
+
+### Loading specific splits from HuggingFace
+
+You can also manually specify splits you want to include as a parameters and it'll return a dictionary where keys are splits that you specified:
+
+```python
+code_alpaca = dl.from_huggingface(
+ "HuggingFaceH4/CodeAlpaca_20K",
+ split = ["train", "test"],
+)
+
+print(f"Splits in dataset: {code_alpaca.keys()}")
+```
+
+If you specify a single split then dataloader will return a List of `dspy.Example` instead of dictionary:
+
+```python
+code_alpaca = dl.from_huggingface(
+ "HuggingFaceH4/CodeAlpaca_20K",
+ split = "train",
+)
+
+print(f"Number of examples in split: {len(code_alpaca)}")
+```
+
+You can slice the split just like you do with HuggingFace Dataset too:
+
+```python
+code_alpaca_80 = dl.from_huggingface(
+ "HuggingFaceH4/CodeAlpaca_20K",
+ split = "train[:80%]",
+)
+
+print(f"Number of examples in split: {len(code_alpaca_80)}")
+
+code_alpaca_20_80 = dl.from_huggingface(
+ "HuggingFaceH4/CodeAlpaca_20K",
+ split = "train[20%:80%]",
+)
+
+print(f"Number of examples in split: {len(code_alpaca_20_80)}")
+```
+
+### Loading specific subset from HuggingFace
+
+If a dataset has a subset you can pass it as an arg like you do with `load_dataset` in HuggingFace:
+
+```python
+gms8k = dl.from_huggingface(
+ "gsm8k",
+ "main",
+ input_keys = ("question",),
+)
+
+print(f"Keys present in the returned dict: {list(gms8k.keys())}")
+
+print(f"Number of examples in train set: {len(gms8k['train'])}")
+print(f"Number of examples in test set: {len(gms8k['test'])}")
+```
+
+### Loading from CSV
+
+```python
+dolly_100_dataset = dl.from_csv("dolly_subset_100_rows.csv",)
+```
+
+You can choose only selected columns from the csv by specifying them in the arguments:
+
+```python
+dolly_100_dataset = dl.from_csv(
+ "dolly_subset_100_rows.csv",
+ fields=["instruction", "context", "response"],
+ input_keys=("instruction", "context")
+)
+```
+
+### Splitting a List of `dspy.Example`
+
+```python
+splits = dl.train_test_split(dataset, train_size=0.8) # `dataset` is a List of dspy.Example
+train_dataset = splits['train']
+test_dataset = splits['test']
+```
+
+### Sampling from List of `dspy.Example`
+
+```python
+sampled_example = dl.sample(dataset, n=5) # `dataset` is a List of dspy.Example
+```
## DSPy Programs.
diff --git a/docs/deep-dive/language_model_clients/remote_models/Together.mdx b/docs/deep-dive/language_model_clients/remote_models/Together.mdx
index 0ae60e45a..1b94de9b4 100644
--- a/docs/deep-dive/language_model_clients/remote_models/Together.mdx
+++ b/docs/deep-dive/language_model_clients/remote_models/Together.mdx
@@ -1,3 +1,5 @@
+import AuthorDetails from '@site/src/components/AuthorDetails';
+
## Togetherß
#### Adapted from documentation provided by https://github.com/insop