Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

feat(retrieve): add support for faiss retriever, update setup.md, docs

Jagane Sundar7e1212c8f75f
diff --git a/docs/retrieval_models_client.md b/docs/retrieval_models_client.md
index b1a0e4e51..8f31e0bbd 100644
--- a/docs/retrieval_models_client.md
+++ b/docs/retrieval_models_client.md
@@ -9,6 +9,7 @@ This documentation provides an overview of the DSPy Retrieval Model Clients.
| ColBERTv2 | [ColBERTv2 Section](#ColBERTv2) |
| AzureCognitiveSearch | [AzureCognitiveSearch Section](#AzureCognitiveSearch) |
| ChromadbRM | [ChromadbRM Section](#ChromadbRM) |
+| Faiss | [Faiss Section](#Faiss) |
## ColBERTv2
@@ -155,3 +156,62 @@ Search the chromadb collection for the top `k` passages matching the given query
**Returns:**
- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with a `long_text` attribute.
+
+## Faiss
+
+### Quickstart with the default vectorizer
+
+The **FaissRM** module provides a retriever that uses an in-memory Faiss vector database. This module does not include a vectorizer; instead it supports any subclass of **dsp.modules.sentence_vectorizer.BaseSentenceVectorizer**. If a vectorizer is not provided, an instance of **dsp.modules.sentence_vectorizer.SentenceTransformersVectorizer** is created and used by **FaissRM**. Note that the default embedding model for **SentenceTransformersVectorizer** is **all-MiniLM-L6-v2**
+
+
+```python
+import dspy
+from dspy.retrieve import faiss_rm
+
+document_chunks = [
+ "The superbowl this year was played between the San Francisco 49ers and the Kanasas City Chiefs",
+ "Pop corn is often served in a bowl",
+ "The Rice Bowl is a Chinese Restaurant located in the city of Tucson, Arizona",
+ "Mars is the fourth planet in the Solar System",
+ "An aquarium is a place where children can learn about marine life",
+ "The capital of the United States is Washington, D.C",
+ "Rock and Roll musicians are honored by being inducted in the Rock and Roll Hall of Fame",
+ "Music albums were published on Long Play Records in the 70s and 80s",
+ "Sichuan cuisine is a spicy cuisine from central China",
+ "The interest rates for mortgages is considered to be very high in 2024",
+]
+
+frm = faiss_rm.FaissRM(document_chunks)
+turbo = dspy.OpenAI(model="gpt-3.5-turbo")
+dspy.settings.configure(lm=turbo, rm=frm)
+print(frm(["I am in the mood for Chinese food"]))
+```
+
+### Constructor
+
+Initialize an instance of FaissRM by providing it with a vectorizer and a list of strings
+
+```python
+FaissRM(
+ document_chunks: List[str],
+ vectorizer: dsp.modules.sentence_vectorizer.BaseSentenceVectorizer,
+ k: int = 3
+)
+```
+
+**Parameters:**
+- `document_chunks` (_List[str]_): a list of strings that comprises the corpus to search. You cannot add/insert/upsert to this list after creating this FaissRM object.
+- `vectorizer` (_dsp.modules.sentence_vectorizer.BaseSentenceVectorizer_, _optional_): If not provided, a dsp.modules.sentence_vectorizer.SentenceTransformersVectorizer object is created and used.
+- `k` (_int_, _optional_): The number of top passages to retrieve. Defaults to 3.
+
+### Methods
+
+#### `forward(self, query_or_queries: Union[str, List[str]]) -> dspy.Prediction`
+
+Search the FaissRM vector database for the top `k` passages matching the given query or queries, using embeddings generated via the vectorizer specified at FaissRM construction time
+
+**Parameters:**
+- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.
+
+**Returns:**
+- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with a `long_text` attribute and an `index` attribute. The `index` attribute is the index in the document_chunks array provided to this FaissRM object at construction time.
diff --git a/dspy/retrieve/faiss_rm.py b/dspy/retrieve/faiss_rm.py
new file mode 100644
index 000000000..7a74ec4b9
--- /dev/null
+++ b/dspy/retrieve/faiss_rm.py
@@ -0,0 +1,156 @@
+"""Retriever model for faiss: https://github.com/facebookresearch/faiss.
+Author: Jagane Sundar: https://github.com/jagane.
+"""
+
+import logging
+from typing import Union
+
+import numpy as np
+
+import dspy
+from dsp.modules.sentence_vectorizer import SentenceTransformersVectorizer
+from dsp.utils import dotdict
+
+try:
+ import faiss
+except ImportError:
+ faiss = None
+
+if faiss is None:
+ raise ImportError(
+ """
+ The faiss package is required. Install it using `pip install dspy-ai[faiss-cpu]`
+ """,
+ )
+
+
+class FaissRM(dspy.Retrieve):
+ """A retrieval module that uses an in-memory Faiss to return the top passages for a given query.
+
+ Args:
+ document_chunks: the input text chunks
+ vectorizer: an object that is a subclass of BaseSentenceVectorizer
+ k (int, optional): The number of top passages to retrieve. Defaults to 3.
+
+ Returns:
+ dspy.Prediction: An object containing the retrieved passages.
+
+ Examples:
+ Below is a code snippet that shows how to use this as the default retriver:
+ ```python
+ import dspy
+ from dspy.retrieve import faiss_rm
+
+ document_chunks = [
+ "The superbowl this year was played between the San Francisco 49ers and the Kanasas City Chiefs",
+ "Pop corn is often served in a bowl",
+ "The Rice Bowl is a Chinese Restaurant located in the city of Tucson, Arizona",
+ "Mars is the fourth planet in the Solar System",
+ "An aquarium is a place where children can learn about marine life",
+ "The capital of the United States is Washington, D.C",
+ "Rock and Roll musicians are honored by being inducted in the Rock and Roll Hall of Fame",
+ "Music albums were published on Long Play Records in the 70s and 80s",
+ "Sichuan cuisine is a spicy cuisine from central China",
+ "The interest rates for mortgages is considered to be very high in 2024",
+ ]
+
+ frm = faiss_rm.FaissRM(document_chunks)
+ turbo = dspy.OpenAI(model="gpt-3.5-turbo")
+ dspy.settings.configure(lm=turbo, rm=frm)
+ print(frm(["I am in the mood for Chinese food"]))
+ ```
+
+ Below is a code snippet that shows how to use this in the forward() function of a module
+ ```python
+ self.retrieve = FaissRM(k=num_passages)
+ ```
+ """
+
+ def __init__(self, document_chunks, vectorizer=None, k: int = 3):
+ """Inits the faiss retriever.
+
+ Args:
+ document_chunks: a list of input strings.
+ vectorizer: an object that is a subclass of BaseTransformersVectorizer.
+ k: number of matches to return.
+ """
+ if vectorizer:
+ self._vectorizer = vectorizer
+ else:
+ self._vectorizer = SentenceTransformersVectorizer()
+ embeddings = self._vectorizer(document_chunks)
+ xb = np.array(embeddings)
+ d = len(xb[0])
+ logging.info(f"FaissRM: embedding size={d}")
+ if len(xb) < 100:
+ self._faiss_index = faiss.IndexFlatL2(d)
+ self._faiss_index.add(xb)
+ else:
+ # if we have at least 100 vectors, we use Voronoi cells
+ nlist = 100
+ quantizer = faiss.IndexFlatL2(d)
+ self._faiss_index = faiss.IndexIVFFlat(quantizer, d, nlist)
+ self._faiss_index.train(xb)
+ self._faiss_index.add(xb)
+
+ logging.info(f"{self._faiss_index.ntotal} vectors in faiss index")
+ self._document_chunks = document_chunks # save the input document chunks
+
+ super().__init__(k=k)
+
+ def _dump_raw_results(self, queries, index_list, distance_list) -> None:
+ for i in range(len(queries)):
+ indices = index_list[i]
+ distances = distance_list[i]
+ logging.debug(f"Query: {queries[i]}")
+ for j in range(len(indices)):
+ logging.debug(f" Hit {j} = {indices[j]}/{distances[j]}: {self._document_chunks[indices[j]]}")
+ return
+
+ def forward(self, query_or_queries: Union[str, list[str]]) -> dspy.Prediction:
+ """Search the faiss index for self.k top passages for query.
+
+ Args:
+ query_or_queries (Union[str, List[str]]): The query or queries to search for.
+
+ Returns:
+ dspy.Prediction: An object containing the retrieved passages.
+ """
+ queries = [query_or_queries] if isinstance(query_or_queries, str) else query_or_queries
+ queries = [q for q in queries if q] # Filter empty queries
+ embeddings = self._vectorizer(queries)
+ emb_npa = np.array(embeddings)
+ # For single query, just look up the top k passages
+ if len(queries) == 1:
+ distance_list, index_list = self._faiss_index.search(emb_npa, self.k)
+ # self._dump_raw_results(queries, index_list, distance_list)
+ passages = [(self._document_chunks[ind], ind) for ind in index_list[0]]
+ passages = [dotdict({"long_text": passage[0], "index": passage[1]}) for passage in passages]
+ return dspy.Prediction(passages=passages)
+
+ distance_list, index_list = self._faiss_index.search(emb_npa, self.k * 3)
+ # self._dump_raw_results(queries, index_list, distance_list)
+ passage_scores = {}
+ for emb in range(len(embeddings)):
+ indices = index_list[emb] # indices of neighbors for embeddings[emb] - this is an array of k*3 integers
+ distances = distance_list[
+ emb
+ ] # distances of neighbors for embeddings[emb] - this is an array of k*3 floating point numbers
+ for res in range(self.k * 3):
+ neighbor = indices[res]
+ distance = distances[res]
+ if neighbor in passage_scores:
+ passage_scores[neighbor].append(distance)
+ else:
+ passage_scores[neighbor] = [distance]
+ # Note re. sorting:
+ # first degree sort: number of queries that got a hit with any particular document chunk. More
+ # is a better match. This is len(queries)-len(x[1])
+ # second degree sort: sum of the distances of each hit returned by faiss. Smaller distance is a better match
+ sorted_passages = sorted(passage_scores.items(), key=lambda x: (len(queries) - len(x[1]), sum(x[1])))[: self.k]
+ return dspy.Prediction(
+ passages=[
+ dotdict({"long_text": self._document_chunks[passage_index], "index": passage_index})
+ for passage_index, _ in sorted_passages
+ ],
+ )
diff --git a/setup.py b/setup.py
index 6c5175e60..4e6bd9b4d 100644
--- a/setup.py
+++ b/setup.py
@@ -28,6 +28,7 @@ setup(
"mongodb": ["pymongo~=3.12.0"],
"pinecone": ["pinecone-client~=2.2.4"],
"weaviate": ["weaviate-client~=3.26.1"],
+ "faiss-cpu": ["sentence_transformers", "faiss-cpu"],
},
classifiers=[
"Development Status :: 3 - Alpha",