Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Fix chat adapter finetune (#8044)

Dilara Soylu53045805e686
diff --git a/docs/docs/tutorials/classification_finetuning/index.ipynb b/docs/docs/tutorials/classification_finetuning/index.ipynb
index 17120ed87..d7c7f4300 100644
--- a/docs/docs/tutorials/classification_finetuning/index.ipynb
+++ b/docs/docs/tutorials/classification_finetuning/index.ipynb
@@ -12,16 +12,25 @@
"\n",
"### Install dependencies and download data\n",
"\n",
- "Install the latest DSPy via `pip install -U --pre dspy` and follow along. This tutorial depends on DSPy 2.6.0 (pre-release).\n",
+ "Install the latest DSPy via `pip install -U dspy>=2.6.0` and follow along (or `uv pip`, if you prefer). This tutorial depends on DSPy >= 2.6.0.\n",
"\n",
"This tutorial requires a local GPU at the moment for inference, though we plan to support ollama serving for finetuned models as well.\n",
"\n",
"You will also need the following dependencies:\n",
+ "1. Inference: We use SGLang to run local inference servers. You can install the latest version by following the instructions here: https://docs.sglang.ai/start/install.html\n",
+ "Shared below is the most recent install command as of 04/02/2025, but we recommend that you follow the instructions in the most up to date version by navigating to the installation link.\n",
+ "This ensures that the fine-tuning packages and the `sglang` package are in sync.\n",
+ " ```shell\n",
+ " > pip install --upgrade pip\n",
+ " > pip install uv\n",
+ " > uv pip install \"sglang[all]>=0.4.4.post3\" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python\n",
+ " ```\n",
+ "1. Fine-tuning: We use the following packages. Note that we specify the version for the transformers package as a temporary fix to a recent issue: https://github.com/huggingface/trl/issues/2338\n",
+ " ```shell\n",
+ " > uv pip install -U torch transformers==4.48.3 accelerate trl peft\n",
+ " ```\n",
"\n",
- "```shell\n",
- "> pip install \"sglang[all]\"; pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/\n",
- "> pip install -U torch transformers accelerate trl peft\n",
- "```"
+ "We recommend using `uv` package manager to speed up the installation."
]
},
{
@@ -241,15 +250,39 @@
"\n",
"On every question in the (unlabeled) training set, this will invoke the teacher program, which will produce reasoning and select a class. This will be traced and then constitute a training set for all modules (in this case, just the one CoT module) in the student program.\n",
"\n",
+ "When the `compile` method is called, the `BootstrapFinetune` optimizer will use the passed teacher program (or programs, you can pass a list!) to create a training dataset.\n",
+ "It will then use this training dataset to create a fine-tuned version of the LM set for the `student` program, replacing it with the trained LM.\n",
+ "Note that the trained LM will be a new LM instance (the `student_lm` object we instantiated here will be untouched!)\n",
+ "\n",
"Note: If you have labels, you can pass `metric` to the constructor of `BootstrapFinetune`. If you want to apply this in practice, you can pass `train_kwargs` to the constructor to control local LM training settings: `device`, `use_peft`, `num_train_epochs`, `per_device_train_batch_size`, `gradient_accumulation_steps`, `learning_rate`, `max_seq_length`, `packing`, `bf16`, and `output_dir`."
]
},
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Optional:\n",
+ "# [1] You can set `DSPY_FINETUNEDIR` environment variable to control where the directory that will be used to store the\n",
+ "# checkpoints and fine-tuning data. If this is not set, `DSPY_CACHEDIR` is used by default.\n",
+ "# [2] You can set the `CUDA_VISIBLE_DEVICES` environment variable to control the GPU that will be used for fine-tuning\n",
+ "# and inference. If this is not set and the default GPU that's used by HuggingFace's `transformers` library is\n",
+ "# occupied, an OutOfMemoryError might be raised.\n",
+ "#\n",
+ "# import os\n",
+ "# os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n",
+ "# os.environ[\"DSPY_FINETUNEDIR\"] = \"/path/to/dir\""
+ ]
+ },
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
+ "dspy.settings.experimental = True # fine-tuning is an experimental feature, so we set a flag to enable it\n",
+ "\n",
"optimizer = dspy.BootstrapFinetune(num_threads=16) # if you *do* have labels, pass metric=your_metric here!\n",
"classify_ft = optimizer.compile(student_classify, teacher=teacher_classify, trainset=unlabeled_trainset)"
]
@@ -1088,7 +1121,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.11.9"
+ "version": "3.11.11"
}
},
"nbformat": 4,
diff --git a/dspy/adapters/chat_adapter.py b/dspy/adapters/chat_adapter.py
index abb381f96..98ed24aa2 100644
--- a/dspy/adapters/chat_adapter.py
+++ b/dspy/adapters/chat_adapter.py
@@ -193,18 +193,26 @@ class ChatAdapter(Adapter):
return "\n\n".join(output).strip()
- # TODO(PR): Looks ok?
def format_finetune_data(
- self, signature: Type[Signature], demos: list[dict[str, Any]], inputs: dict[str, Any], outputs: dict[str, Any]
+ self,
+ signature: Type[Signature],
+ demos: list[dict[str, Any]],
+ inputs: dict[str, Any],
+ outputs: dict[str, Any],
) -> dict[str, list[Any]]:
- # Get system + user messages
- messages = self.format(signature, demos, inputs)
-
- # Add the assistant message
- role = "assistant"
- incomplete = False
- assistant_message = self.format_turn(signature, outputs, role, incomplete)
- messages.append(assistant_message)
+ """
+ Format the call data into finetuning data according to the OpenAI API specifications.
- # Wrap the messages in a dictionary with a "messages" key
+ For the chat adapter, this means formatting the data as a list of messages, where each message is a dictionary
+ with a "role" and "content" key. The role can be "system", "user", or "assistant". Then, the messages are
+ wrapped in a dictionary with a "messages" key.
+ """
+ system_user_messages = self.format( # returns a list of dicts with the keys "role" and "content"
+ signature=signature, demos=demos, inputs=inputs
+ )
+ assistant_message_content = self.format_assistant_message_content( # returns a string, without the role
+ signature=signature, outputs=outputs
+ )
+ assistant_message = dict(role="assistant", content=assistant_message_content)
+ messages = system_user_messages + [assistant_message]
return dict(messages=messages)
diff --git a/dspy/clients/lm_local.py b/dspy/clients/lm_local.py
index d74d079a6..8329db978 100644
--- a/dspy/clients/lm_local.py
+++ b/dspy/clients/lm_local.py
@@ -31,8 +31,8 @@ class LocalProvider(Provider):
import sglang # noqa: F401
except ImportError:
raise ImportError(
- "For local model launching, please install sglang by running "
- '`pip install "sglang[all]"; pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/`'
+ "For local model launching, please install sglang."
+ "Navigate to https://docs.sglang.ai/start/install.html for the latest installation instructions!"
)
if hasattr(lm, "process"):
@@ -153,6 +153,7 @@ class LocalProvider(Provider):
raise ValueError("Only chat models are supported for local finetuning.")
data_path = save_data(train_data)
+ logger.info(f"Train data saved to {data_path}")
output_dir = create_output_dir(model, data_path)
default_train_kwargs = {