DSPy: The framework for programming—not prompting—language models
Python
This PR adds [chromadb](https://docs.trychroma.com/getting-started) to the RM.
Insop Songbf14c3c616a4
diff --git a/README.md b/README.mdindex a38564c99..e710c99fa 100644--- a/README.md+++ b/README.md@@ -70,6 +70,12 @@ For the optional Qdrant retrieval integration, include the `qdrant` extra:pip install dspy-ai[qdrant]```+For the optional [chromadb](https://github.com/chroma-core/chroma) retrieval integration, include the `chromadb` extra:++```+pip install dspy-ai[chromadb]+```+## 2) Syntax: You're in charge of the workflow—it's free-form Python code!**DSPy** hides tedious prompt engineering, but it cleanly exposes the important decisions you need to make: **[1]** what's your system design going to look like? **[2]** what are the important constraints on the behavior of your program?diff --git a/dspy/retrieve/chromadb_rm.py b/dspy/retrieve/chromadb_rm.pyindex b6f515a9b..82ab0023f 100644--- a/dspy/retrieve/chromadb_rm.py+++ b/dspy/retrieve/chromadb_rm.py@@ -11,8 +11,6 @@ from dsp.utils import dotdicttry:import chromadbfrom chromadb.config import Settings-- # from chromadb.utils import embedding_functionsexcept ImportError:chromadb = None@@ -27,7 +25,7 @@ class ChromadbRM(dspy.Retrieve):A retrieval module that uses chromadb to return the top passages for a given query.Assumes that the chromadb index has been created and populated with the following metadata:- - text: The text of the passage+ - documents: The text of the passageArgs:collection_name (str): chromadb collection name@@ -41,16 +39,18 @@ class ChromadbRM(dspy.Retrieve):dspy.Prediction: An object containing the retrieved passages.Examples:- Below is a code snippet that shows how to use this as the default retriver:+ Below is a code snippet that shows how to use this as the default retriever:```pythonllm = dspy.OpenAI(model="gpt-3.5-turbo")- retriever_model = ChromadbRM('collection_name', 'db_path', openai.api_key, 'azure')+ retriever_model = ChromadbRM('collection_name', 'db_path')dspy.settings.configure(lm=llm, rm=retriever_model)+ # to test the retriever with "my query"+ retriever_model("my query")```Below is a code snippet that shows how to use this in the forward() function of a module```python- self.retrieve = ChromadbRM(k=num_passages)+ self.retrieve = ChromadbRM('collection_name', 'db_path', k=num_passages)```"""@@ -66,15 +66,11 @@ class ChromadbRM(dspy.Retrieve):openai_api_version: Optional[str] = None,k: int = 7,):- # self._openai_embed_model = embedding_functions.OpenAIEmbeddingFunction(- # model_name=openai_embed_model- # )-self._openai_embed_model = openai_embed_modelself._init_chromadb(collection_name, persist_directory)- # If not provided, defaults to env vars OPENAI_API_KEY and OPENAI_ORGANIZATION+ # If not provided, defaults to env varsif openai_api_key:openai.api_key = openai_api_keyif openai_api_type:@@ -171,8 +167,6 @@ class ChromadbRM(dspy.Retrieve):query_embeddings=embeddings, n_results=k)- # 'metadatas': [[{'page': 30, 'source': "..."}, {...}]]passages = [dotdict({"long_text": x}) for x in results["documents"][0]]- print(passages)return passages