Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Update Dataloaders Docs

Herumb Shandilya90652e105605
diff --git a/examples/dataloaders/dataloaders_dolly.ipynb b/examples/dataloaders/dataloaders_dolly.ipynb
index bdb0faea8..eb230fd0e 100644
--- a/examples/dataloaders/dataloaders_dolly.ipynb
+++ b/examples/dataloaders/dataloaders_dolly.ipynb
@@ -35,11 +35,7 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "XZ2MimQMkyjA",
- "outputId": "dc046d83-1dc7-437b-ea34-201a62efb072"
+ "id": "XZ2MimQMkyjA"
},
"outputs": [],
"source": [
@@ -75,19 +71,19 @@
"id": "PtBml664suSx"
},
"source": [
- "# Using `DataLoader`\n",
+ "# Data Loading Using `DataLoader`\n",
"\n",
"Currently `DataLoader` provides support for data loading for following sources:\n",
"\n",
"* HuggingFace using `from_huggingface()` method.\n",
"* CSV Files using `from_csv()` method.\n",
"\n",
- "We'll start by initializing the `DataLoader` object that takes same argument as that of [`Dataset`](https://github.com/stanfordnlp/dspy/blob/main/dspy/datasets/dataset.py) class which are dataset size (in % format) and seed for each of the three splits: `train`, `dev` and `test`."
+ "We'll start by initializing the `DataLoader` object that takes no arguments, so we can just create an object without arguments."
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 12,
"metadata": {
"id": "UeB7AEvhshfe"
},
@@ -96,11 +92,7 @@
"import dspy\n",
"from dspy.datasets import DataLoader\n",
"\n",
- "dl = DataLoader(\n",
- " train_size = 0.5,\n",
- " dev_size = 0.3,\n",
- " test_size = 0.2,\n",
- ")"
+ "dl = DataLoader()"
]
},
{
@@ -111,27 +103,32 @@
"source": [
"## Loading from HuggingFace\n",
"\n",
- "To load data from Hugging Face we have `from_huggingface` method which loads a dataset from Hugging Face's datasets library and processes it according to the object's configuration.\n",
+ "To load data from Hugging Face we have `from_huggingface` method which loads a dataset from Hugging Face's datasets library and processes it according to the split configuration.\n",
"\n",
- "Parameters:\n",
- "* `dataset_name`: The name of the dataset to load from Hugging Face's datasets.\n",
- "* `fields`: An optional list of strings specifying which fields to include in the final datasets. If None, all fields in the dataset are included.\n",
- "* `splits`: A string or list of strings specifying which splits of the dataset to load. If None, the entire dataset is loaded. If it's a list of splits, **it'll concatinate all the splits into 1 split**.\n",
- "* `revision`: An optional string specifying the version of the dataset to load. Same parameter as that of datasets library."
+ "**Parameters:**\n",
+ "* `dataset_name (str)`: The name of the dataset to load.\n",
+ "* `*args`: Positional arguments passed to the load_dataset function.\n",
+ "* `input_keys (Tuple[str], optional)`: Tuple of input keys to be used in the dataset.\n",
+ "* `fields (Tuple[str], optional)`: Tuple of fields to include from the dataset. If None, all fields are included.\n",
+ "* `**kwargs`: Keyword arguments passed to the load_dataset function.\n",
+ "\n",
+ "**Returns:**\n",
+ "\n",
+ "* A dictionary mapping split names to lists of `dspy.Example` objects, or a list of `dspy.Example` objects if split is a string."
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 25,
"metadata": {
"id": "OslWdJbMvVYU"
},
"outputs": [],
"source": [
- "dolly_dataset = dl.from_huggingface(\n",
- " \"databricks/databricks-dolly-15k\",\n",
- " fields=[\"instruction\", \"context\", \"response\"],\n",
- " splits=\"train\" # or splits = [\"train\", \"test\"]\n",
+ "code_alpaca = dl.from_huggingface(\n",
+ " \"HuggingFaceH4/CodeAlpaca_20K\",\n",
+ " split = [\"train\", \"test\"], # returns Dictionary with train, test keys\n",
+ " input_keys = (\"prompt\",),\n",
")"
]
},
@@ -141,26 +138,160 @@
"id": "RRyYfcTUF121"
},
"source": [
- "Once the above code is executed the data would be returned in a dict with keys `train`, `dev`, and `test` containing the List of `Example` for each split based on the configuration of size and seed set in the object during initialization:"
+ "Once the above code is executed the data would be returned in a dict with keys `train`, and `test` each containing the List of `Example` for each split based on the dataset:"
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 26,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "CIj3Tkj6zWTq",
- "outputId": "d9529450-1232-4c70-b8d8-c5171ea9ab43"
+ "outputId": "2d8bd9bb-566a-41ec-b0ee-fa225514d1c7"
},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Keys present in the returned dict: ['train', 'test']\n",
+ "Number of examples in train set: 18019\n",
+ "Number of examples in test set: 2003\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(f\"Keys present in the returned dict: {list(code_alpaca.keys())}\")\n",
+ "\n",
+ "print(f\"Number of examples in train set: {len(code_alpaca['train'])}\")\n",
+ "print(f\"Number of examples in test set: {len(code_alpaca['test'])}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "NAqtNAUwuKvD"
+ },
+ "source": [
+ "You can also pass a single split or a read instruction or anything that `load_dataset` allows you to pass and the conversion to `dspy.Example` would be taken care of by Dataloader."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "1LX1B8qUlW2E",
+ "outputId": "0b6160e7-7839-4a79-e820-08e4e6fcd80c"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Number of examples in split: 18019\n"
+ ]
+ }
+ ],
+ "source": [
+ "code_alpaca = dl.from_huggingface(\n",
+ " \"HuggingFaceH4/CodeAlpaca_20K\",\n",
+ " split = \"train\",\n",
+ ")\n",
+ "\n",
+ "print(f\"Number of examples in split: {len(code_alpaca)}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9Y14ZZ0Vlh_6"
+ },
+ "source": [
+ "As you can see if the `split` argumnent is a string it will return a List of `dspy.Example` instead of mapping because there is only one split in it. However, HF Dataset can create slices of dataset too and the same concept can be applied here too!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "GpS-zSTUlhXq",
+ "outputId": "cedb796e-e315-4a67-c8ed-ca6dd1082ce1"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Number of examples in split: 14415\n",
+ "Number of examples in split: 10811\n"
+ ]
+ }
+ ],
+ "source": [
+ "code_alpaca_80 = dl.from_huggingface(\n",
+ " \"HuggingFaceH4/CodeAlpaca_20K\",\n",
+ " split = \"train[:80%]\",\n",
+ ")\n",
+ "\n",
+ "print(f\"Number of examples in split: {len(code_alpaca_80)}\")\n",
+ "\n",
+ "code_alpaca_20_80 = dl.from_huggingface(\n",
+ " \"HuggingFaceH4/CodeAlpaca_20K\",\n",
+ " split = \"train[20%:80%]\",\n",
+ ")\n",
+ "\n",
+ "print(f\"Number of examples in split: {len(code_alpaca_20_80)}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jc6IDdc8mSbU"
+ },
+ "source": [
+ "Cool right! In the same way we can use `ReadInstruction` to create even more complicated splits. That aside, if you need to load a dataset from a subset(if any) you can pass it as a positional argument to the method. Similar to how you do it in `load_dataset`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "iVYAFxQAmR1M",
+ "outputId": "c6fbcf18-0029-4355-b359-2bc06e604428"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Keys present in the returned dict: ['train', 'test']\n",
+ "Number of examples in train set: 7473\n",
+ "Number of examples in test set: 1319\n"
+ ]
+ }
+ ],
"source": [
- "print(f\"Keys present in the returned dict: {list(dolly_dataset.keys())}\")\n",
+ "gms8k = dl.from_huggingface(\n",
+ " \"gsm8k\",\n",
+ " \"main\",\n",
+ " input_keys = (\"question\",),\n",
+ ")\n",
+ "\n",
+ "print(f\"Keys present in the returned dict: {list(gms8k.keys())}\")\n",
"\n",
- "print(f\"Number of examples in train set: {len(dolly_dataset['train'])}\")\n",
- "print(f\"Number of examples in dev set: {len(dolly_dataset['dev'])}\")\n",
- "print(f\"Number of examples in test set: {len(dolly_dataset['test'])}\")"
+ "print(f\"Number of examples in train set: {len(gms8k['train'])}\")\n",
+ "print(f\"Number of examples in test set: {len(gms8k['test'])}\")"
]
},
{
@@ -171,24 +302,25 @@
"source": [
"## Loading from CSV\n",
"\n",
- "To load data from CSV we have `from_csv` method which loads a dataset from CSV files and processes it according to the object's configuration just like `from_huggingface` method. Everything is the same as `from_huggingface` method except the arguments which are slightly different but not too much:\n",
+ "The from_csv function in the DataLoader class makes it easy to load data from CSV files for machine learning projects. You just need to tell it where your CSV file is and which parts of the data you're interested in by listing the columns you want. This function turns your CSV data into a list of examples that you can use right away with DSPy modules.\n",
"\n",
- "* `file_path`: The file path of the CSV file to load.\n",
- "* `fields`: An optional list of strings specifying which columns to include in the final datasets. If None, all columns in the dataset are included."
+ "**Parameters:**\n",
+ "* `file_path (str)`: The path to the CSV file.\n",
+ "* `fields (List[str], optional)`: List of fields to include from the dataset. If None, all fields are included.\n",
+ "* `input_keys (Tuple[str], optional)`: Tuple of input keys to be used in the dataset.\n",
+ "\n",
+ "**Returns:**\n",
+ "* A list of `dspy.Example` objects."
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 24,
"metadata": {
"id": "tkK14QOVQwph"
},
"outputs": [],
"source": [
- "dl = DataLoader(\n",
- " train_size = 1.0, # Must be float\n",
- ")\n",
- "\n",
"dolly_100_dataset = dl.from_csv(\n",
" \"dolly_subset_100_rows.csv\",\n",
" fields=[\"instruction\", \"context\", \"response\"],\n",
@@ -206,34 +338,166 @@
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "TIrNy_wrQ4B5",
- "outputId": "0d44d6b5-b72e-48af-939b-3938060e5b37"
+ "outputId": "49e4c90b-e5af-4f86-f865-a553f7f520a7"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Number of examples in train set: 100\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(f\"Number of examples in train set: {len(dolly_100_dataset)}\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "r4K2TWkRWdDR",
+ "outputId": "1ac2cd38-4ecf-452d-ef91-b6f25fd45b65"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Example({'instruction': 'Which of these animals can be pets? Wolf, dog, rabbit, squirrel, cat, lion.', 'context': None, 'response': 'While most animals can be domesticated, dogs, rabbits and cats are common pets.'}) (input_keys={()})"
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "dolly_100_dataset[-1]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "NkCG01E5p6VX"
+ },
+ "source": [
+ "# Manipulating Dataset with `DataLoader`"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "jEEt4kI8qBkQ"
},
- "outputs": [],
"source": [
- "print(f\"Keys present in the returned dict: {list(dolly_100_dataset.keys())}\")\n",
+ "## Sampling Chunks\n",
"\n",
- "print(f\"Number of examples in train set: {len(dolly_100_dataset['train'])}\")"
+ "To sample chunks of data from your dataset, you can use the sample method of the DataLoader class. This method allows you to randomly pick a specified number of examples from your dataset. For example, if you want to sample 5 examples from a dataset `gsm8k` we loaded above, you can use the following code:"
]
},
{
"cell_type": "code",
- "execution_count": null,
+ "execution_count": 32,
"metadata": {
- "id": "r4K2TWkRWdDR"
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "DOv929yUhRT7",
+ "outputId": "5156fbd9-ae98-4414-cfcb-ced25a29e01d"
},
- "outputs": [],
- "source": []
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Number of examples in sampled chunk: 5\n"
+ ]
+ }
+ ],
+ "source": [
+ "sampled_example = dl.sample(dataset=gms8k[\"train\"], n=5)\n",
+ "\n",
+ "print(f\"Number of examples in sampled chunk: {len(sampled_example)}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "3IKtN0Qfyhiu"
+ },
+ "source": [
+ "This will give you a random set of 5 examples from `gms8k` dataset's `train` split that you can use for quick testing or analysis."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "hZyOGFjzqERb"
+ },
+ "source": [
+ "## Splitting Dataset\n",
+ "\n",
+ "To split your dataset into training and testing sets, the DataLoader class provides the `train_test_split` method. This method divides your dataset based on the proportions you specify. For instance, if you have a dataset `dolly_100_dataset` and you want to split it with 80% of the data for training and 20% for testing, you can use the code like this:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 34,
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "id": "djGamczdqIjn",
+ "outputId": "a2fe4161-df45-48bb-dd87-d9abf1e6f6a1"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Number of total examples in dataset: 100\n",
+ "Number of examples in train set: 80\n",
+ "Number of examples in test set: 20\n"
+ ]
+ }
+ ],
+ "source": [
+ "splits = dl.train_test_split(dataset=dolly_100_dataset, train_size=0.8)\n",
+ "train_dataset = splits['train']\n",
+ "test_dataset = splits['test']\n",
+ "\n",
+ "print(f\"Number of total examples in dataset: {len(dolly_100_dataset)}\")\n",
+ "print(f\"Number of examples in train set: {len(train_dataset)}\")\n",
+ "print(f\"Number of examples in test set: {len(test_dataset)}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "bwDoKE3rytsD"
+ },
+ "source": [
+ "Now, `train_dataset` will contain 80% of the examples for training your model, and `test_dataset` will contain the remaining 20% for testing its performance."
+ ]
}
],
"metadata": {
"colab": {
- "provenance": []
+ "collapsed_sections": [
+ "RqSZB2MqsTAQ"
+ ],
+ "provenance": [],
+ "toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",