DSPy: The framework for programming—not prompting—language models
Python
Remove changes to setup.py, moved notebook with KNN example to tests/ folder.
Igor Kotenkovb492faa4a499
diff --git a/dsp/modules/sentence_vectorizer.py b/dsp/modules/sentence_vectorizer.pyindex f45d3b696..87e83427f 100644--- a/dsp/modules/sentence_vectorizer.py+++ b/dsp/modules/sentence_vectorizer.py@@ -49,7 +49,16 @@ class SentenceTransformersVectorizer(BaseSentenceVectorizer):# this isn't a good practice, but with top-level import the whole DSP# module import will be slow (>5 sec), because SentenceTransformer is doing# it's directory/file-related magic under the hood :(- from sentence_transformers import SentenceTransformer++ try:+ from sentence_transformers import SentenceTransformer+ except ImportError as e:+ raise ImportError(+ "You need to install sentence_transformers library to use pretrained embedders. "+ "Please check the official doc https://www.sbert.net/ "+ "or simply run `pip install sentence-transformers"+ )+self.num_devices, self.is_gpu = determine_devices(max_gpu_devices)self.proxy_device = 'cuda' if self.is_gpu else 'cpu'diff --git a/dsp/utils/ann_utils.py b/dsp/utils/ann_utils.pyindex 1f38cd1c6..7eb997196 100644--- a/dsp/utils/ann_utils.py+++ b/dsp/utils/ann_utils.py@@ -1,7 +1,13 @@from typing import Tuple-import faiss-from faiss import Index+try:+ import faiss+ from faiss import Index+except ImportError as e:+ raise ImportError(+ "You need to install FAISS library to perform ANN/KNN. Please check the official doc: "+ "https://github.com/facebookresearch/faiss/blob/main/INSTALL.md"+ )def determine_devices(max_gpu_devices: int = 0) -> Tuple[int, bool]:diff --git a/setup.py b/setup.pyindex a544e19d7..6db14be91 100644--- a/setup.py+++ b/setup.py@@ -12,12 +12,10 @@ setup(python_requires='>=3.8',install_requires=["backoff",- "faiss-cpu","joblib","jupyter","openai","pandas",- "sentence-transformers","spacy","regex","ujson",diff --git a/knn_demonstrations_test.ipynb b/tests/knn_demonstrations_test.ipynbsimilarity index 79%rename from knn_demonstrations_test.ipynbrename to tests/knn_demonstrations_test.ipynbindex 69e49db6a..799c8d6a2 100644--- a/knn_demonstrations_test.ipynb+++ b/tests/knn_demonstrations_test.ipynb@@ -1,5 +1,27 @@{"cells": [+ {+ "attachments": {},+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## 🎓 DSP + KNN: use simmilar Questions to add better in-context examples\n",+ "\n",+ "This notebook illustrates the KNN-based prompt picking for Demonstrate stage of DSP framework.\n",+ "\n",+ "There are two types of supported embedders:\n",+ "1. with the help of models from `sentence-transformers` library;\n",+ "2. using OpenAI's paid API. \n",+ "\n",+ "For the first option, you need to install `sentence-transformers` using pip (`pip install sentence-transformers`). Please check this [page](https://www.sbert.net/) to check the installation instructions. This library allows you to use a set of pre-trained feature extractors so that it will convert sentences to embeddings under the hood.\n",+ "\n",+ "For both options, you need to install the FAISS library to perform KNN/ANN (Approximate Nearest Neighbor) search.\n",+ "\n",+ "You can install the CPU version of the library with `pip install faiss-cpu`. This is enough for most of the use-cases, especially if your dataset of Questions and Answers is small (<1M samples).\n",+ "\n",+ "For the official instructions (on FAISS with GPU support) and guidelines, please check [this doc page](https://github.com/facebookresearch/faiss/blob/main/INSTALL.md). "+ ]+ },{"cell_type": "code","execution_count": 1,@@ -31,6 +53,18 @@"train = [dsp.Example(question=question, answer=answer) for question, answer in train]"]},+ {+ "attachments": {},+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Case 1: precomputed embeddings\n",+ "\n",+ "If you have precomputed embeddings, you can manually assign a vector for each of the Examples. \n",+ "\n",+ "For the demonstration reason, we will shift some embeddings so that we will know they aren't the closest ones for some queries."+ ]+ },{"cell_type": "code","execution_count": 3,@@ -100,7 +134,8 @@],"source": ["knn_res_dev = knn_func(dev[0], 3)\n",- "[train.index(i) for i in knn_res_dev] # first 3 train examples are the closest, the first one w index 1 is the first in the list"+ "[train.index(i) for i in knn_res_dev]\n",+ "# first 3 train examples are the closest, the first one with index 1 is the first in the list"]},{@@ -121,7 +156,8 @@],"source": ["knn_res_train = knn_func(train[-2], 3)\n",- "[train.index(i) for i in knn_res_train] # as expected"+ "[train.index(i) for i in knn_res_train]\n",+ "# as expected, Eamples 0-1-2 can't be among 3 closest, because they were shifted"]},{@@ -129,7 +165,9 @@"cell_type": "markdown","metadata": {},"source": [- "### SentenceTransformer-based knn"+ "## Case 2: compute embeddings using pre-trained Sentence-Transformers\n",+ "\n",+ "In this case, the model will be downloaded & used to extract features from the question text."]},{@@ -148,7 +186,7 @@" ('In what year was the star of To Hell and Back born?', ['1925']),\n"," ('Which award did the first book of Gary Zukav receive?', ['U.S. National Book Award', 'National Book Award']),\n"," ('What city was the victim of Joseph Druces working in?', ['Boston, Massachusetts', 'Boston']),\n",- " # extra\n",+ " # extra, with the same entities (persons) mentioned\n"," ('Who founded Smart Studios in Madison, while still performing drums in Spooner at night and driving a taxi cab during the day?', ['Butch Vig']),\n"," ('There were two founders of recording studio called \"Smart Studios\". The first one was Butch Vig. Who was the second founder?', ['Steve Marker']),\n","\n",@@ -218,6 +256,7 @@"metadata": {},"outputs": [],"source": [+ "# set new vectorizer for all calls\n","dsp.settings.configure(vectorizer=dsp.SentenceTransformersVectorizer())"]},@@ -249,6 +288,16 @@" print(i)"]},+ {+ "attachments": {},+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "## Case 3: compute embeddings using paid API from OpenAI\n",+ "\n",+ "In this case, we will send Questions to the API endpoint and get embeddings to search among."+ ]+ },{"cell_type": "code","execution_count": 14,