Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Remove changes to setup.py, moved notebook with KNN example to tests/ folder.

Igor Kotenkovb492faa4a499
diff --git a/dsp/modules/sentence_vectorizer.py b/dsp/modules/sentence_vectorizer.py
index f45d3b696..87e83427f 100644
--- a/dsp/modules/sentence_vectorizer.py
+++ b/dsp/modules/sentence_vectorizer.py
@@ -49,7 +49,16 @@ class SentenceTransformersVectorizer(BaseSentenceVectorizer):
# this isn't a good practice, but with top-level import the whole DSP
# module import will be slow (>5 sec), because SentenceTransformer is doing
# it's directory/file-related magic under the hood :(
- from sentence_transformers import SentenceTransformer
+
+ try:
+ from sentence_transformers import SentenceTransformer
+ except ImportError as e:
+ raise ImportError(
+ "You need to install sentence_transformers library to use pretrained embedders. "
+ "Please check the official doc https://www.sbert.net/ "
+ "or simply run `pip install sentence-transformers"
+ )
+
self.num_devices, self.is_gpu = determine_devices(max_gpu_devices)
self.proxy_device = 'cuda' if self.is_gpu else 'cpu'
diff --git a/dsp/utils/ann_utils.py b/dsp/utils/ann_utils.py
index 1f38cd1c6..7eb997196 100644
--- a/dsp/utils/ann_utils.py
+++ b/dsp/utils/ann_utils.py
@@ -1,7 +1,13 @@
from typing import Tuple
-import faiss
-from faiss import Index
+try:
+ import faiss
+ from faiss import Index
+except ImportError as e:
+ raise ImportError(
+ "You need to install FAISS library to perform ANN/KNN. Please check the official doc: "
+ "https://github.com/facebookresearch/faiss/blob/main/INSTALL.md"
+ )
def determine_devices(max_gpu_devices: int = 0) -> Tuple[int, bool]:
diff --git a/setup.py b/setup.py
index a544e19d7..6db14be91 100644
--- a/setup.py
+++ b/setup.py
@@ -12,12 +12,10 @@ setup(
python_requires='>=3.8',
install_requires=[
"backoff",
- "faiss-cpu",
"joblib",
"jupyter",
"openai",
"pandas",
- "sentence-transformers",
"spacy",
"regex",
"ujson",
diff --git a/knn_demonstrations_test.ipynb b/tests/knn_demonstrations_test.ipynb
similarity index 79%
rename from knn_demonstrations_test.ipynb
rename to tests/knn_demonstrations_test.ipynb
index 69e49db6a..799c8d6a2 100644
--- a/knn_demonstrations_test.ipynb
+++ b/tests/knn_demonstrations_test.ipynb
@@ -1,5 +1,27 @@
{
"cells": [
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 🎓 DSP + KNN: use simmilar Questions to add better in-context examples\n",
+ "\n",
+ "This notebook illustrates the KNN-based prompt picking for Demonstrate stage of DSP framework.\n",
+ "\n",
+ "There are two types of supported embedders:\n",
+ "1. with the help of models from `sentence-transformers` library;\n",
+ "2. using OpenAI's paid API. \n",
+ "\n",
+ "For the first option, you need to install `sentence-transformers` using pip (`pip install sentence-transformers`). Please check this [page](https://www.sbert.net/) to check the installation instructions. This library allows you to use a set of pre-trained feature extractors so that it will convert sentences to embeddings under the hood.\n",
+ "\n",
+ "For both options, you need to install the FAISS library to perform KNN/ANN (Approximate Nearest Neighbor) search.\n",
+ "\n",
+ "You can install the CPU version of the library with `pip install faiss-cpu`. This is enough for most of the use-cases, especially if your dataset of Questions and Answers is small (<1M samples).\n",
+ "\n",
+ "For the official instructions (on FAISS with GPU support) and guidelines, please check [this doc page](https://github.com/facebookresearch/faiss/blob/main/INSTALL.md). "
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
@@ -31,6 +53,18 @@
"train = [dsp.Example(question=question, answer=answer) for question, answer in train]"
]
},
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Case 1: precomputed embeddings\n",
+ "\n",
+ "If you have precomputed embeddings, you can manually assign a vector for each of the Examples. \n",
+ "\n",
+ "For the demonstration reason, we will shift some embeddings so that we will know they aren't the closest ones for some queries."
+ ]
+ },
{
"cell_type": "code",
"execution_count": 3,
@@ -100,7 +134,8 @@
],
"source": [
"knn_res_dev = knn_func(dev[0], 3)\n",
- "[train.index(i) for i in knn_res_dev] # first 3 train examples are the closest, the first one w index 1 is the first in the list"
+ "[train.index(i) for i in knn_res_dev]\n",
+ "# first 3 train examples are the closest, the first one with index 1 is the first in the list"
]
},
{
@@ -121,7 +156,8 @@
],
"source": [
"knn_res_train = knn_func(train[-2], 3)\n",
- "[train.index(i) for i in knn_res_train] # as expected"
+ "[train.index(i) for i in knn_res_train]\n",
+ "# as expected, Eamples 0-1-2 can't be among 3 closest, because they were shifted"
]
},
{
@@ -129,7 +165,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### SentenceTransformer-based knn"
+ "## Case 2: compute embeddings using pre-trained Sentence-Transformers\n",
+ "\n",
+ "In this case, the model will be downloaded & used to extract features from the question text."
]
},
{
@@ -148,7 +186,7 @@
" ('In what year was the star of To Hell and Back born?', ['1925']),\n",
" ('Which award did the first book of Gary Zukav receive?', ['U.S. National Book Award', 'National Book Award']),\n",
" ('What city was the victim of Joseph Druces working in?', ['Boston, Massachusetts', 'Boston']),\n",
- " # extra\n",
+ " # extra, with the same entities (persons) mentioned\n",
" ('Who founded Smart Studios in Madison, while still performing drums in Spooner at night and driving a taxi cab during the day?', ['Butch Vig']),\n",
" ('There were two founders of recording studio called \"Smart Studios\". The first one was Butch Vig. Who was the second founder?', ['Steve Marker']),\n",
"\n",
@@ -218,6 +256,7 @@
"metadata": {},
"outputs": [],
"source": [
+ "# set new vectorizer for all calls\n",
"dsp.settings.configure(vectorizer=dsp.SentenceTransformersVectorizer())"
]
},
@@ -249,6 +288,16 @@
" print(i)"
]
},
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Case 3: compute embeddings using paid API from OpenAI\n",
+ "\n",
+ "In this case, we will send Questions to the API endpoint and get embeddings to search among."
+ ]
+ },
{
"cell_type": "code",
"execution_count": 14,