DSPy: The framework for programming—not prompting—language models
Python
fix
dbczumarfc6fc6d6fc06
diff --git a/docs/api/retrieval_model_clients/AzureCognitiveSearch.md b/docs/api/retrieval_model_clients/AzureCognitiveSearch.mdindex 832184563..bb210e830 100644--- a/docs/api/retrieval_model_clients/AzureCognitiveSearch.md+++ b/docs/api/retrieval_model_clients/AzureCognitiveSearch.md@@ -1,5 +1,5 @@----sidebar_position: 2+sidebar_position: 3---# retrieve.AzureCognitiveSearchdiff --git a/docs/api/retrieval_model_clients/ChromadbRM.md b/docs/api/retrieval_model_clients/ChromadbRM.mdindex cfff233d2..97fbe26d4 100644--- a/docs/api/retrieval_model_clients/ChromadbRM.md+++ b/docs/api/retrieval_model_clients/ChromadbRM.md@@ -1,5 +1,5 @@----sidebar_position: 3+sidebar_position: 4---# retrieve.ChromadbRM@@ -18,6 +18,7 @@ ChromadbRM(```**Parameters:**+- `collection_name` (_str_): The name of the chromadb collection.- `persist_directory` (_str_): Path to the directory where chromadb data is persisted.- `embedding_function` (_Optional[EmbeddingFunction[Embeddable]]_, _optional_): The function used for embedding documents and queries. Defaults to `DefaultEmbeddingFunction()` if not specified.@@ -30,10 +31,12 @@ ChromadbRM(Search the chromadb collection for the top `k` passages matching the given query or queries, using embeddings generated via the specified `embedding_function`.**Parameters:**+- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.**Returns:**+- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with schema `[{"id": str, "score": float, "long_text": str, "metadatas": dict }]`### Quickstart with OpenAI Embeddingsdiff --git a/docs/api/retrieval_model_clients/FaissRM.md b/docs/api/retrieval_model_clients/FaissRM.mdindex 634904521..89aed38e7 100644--- a/docs/api/retrieval_model_clients/FaissRM.md+++ b/docs/api/retrieval_model_clients/FaissRM.md@@ -1,5 +1,5 @@----sidebar_position: 4+sidebar_position: 5---# retrieve.FaissRM@@ -17,6 +17,7 @@ FaissRM(```**Parameters:**+- `document_chunks` (_List[str]_): a list of strings that comprises the corpus to search. You cannot add/insert/upsert to this list after creating this FaissRM object.- `vectorizer` (_dsp.modules.sentence_vectorizer.BaseSentenceVectorizer_, _optional_): If not provided, a dsp.modules.sentence_vectorizer.SentenceTransformersVectorizer object is created and used.- `k` (_int_, _optional_): The number of top passages to retrieve. Defaults to 3.@@ -28,16 +29,17 @@ FaissRM(Search the FaissRM vector database for the top `k` passages matching the given query or queries, using embeddings generated via the vectorizer specified at FaissRM construction time**Parameters:**+- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.**Returns:**+- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with a `long_text` attribute and an `index` attribute. The `index` attribute is the index in the document_chunks array provided to this FaissRM object at construction time.### Quickstart with the default vectorizerThe **FaissRM** module provides a retriever that uses an in-memory Faiss vector database. This module does not include a vectorizer; instead it supports any subclass of **dsp.modules.sentence_vectorizer.BaseSentenceVectorizer**. If a vectorizer is not provided, an instance of **dsp.modules.sentence_vectorizer.SentenceTransformersVectorizer** is created and used by **FaissRM**. Note that the default embedding model for **SentenceTransformersVectorizer** is **all-MiniLM-L6-v2**-```pythonimport dspyfrom dspy.retrieve.faiss_rm import FaissRMdiff --git a/docs/api/retrieval_model_clients/MilvusRM.md b/docs/api/retrieval_model_clients/MilvusRM.mdindex e8fb41e42..2f10b77c5 100644--- a/docs/api/retrieval_model_clients/MilvusRM.md+++ b/docs/api/retrieval_model_clients/MilvusRM.md@@ -1,5 +1,5 @@----sidebar_position: 5+sidebar_position: 6---# retrieve.MilvusRM@@ -20,13 +20,14 @@ MilvusRM(```**Parameters:**+- `collection_name (str)`: The name of the Milvus collection to query against.- `uri (str, optional)`: The Milvus connection uri. Defaults to "http://localhost:19530".- `token (str, optional)`: The Milvus connection token. Defaults to None.- `db_name (str, optional)`: The Milvus database name. Defaults to "default".- `embedding_function (callable, optional)`: The function to convert a list of text to embeddings.- The embedding function should take a list of text strings as input and output a list of embeddings.- Defaults to None. By default, it will get OpenAI client by the environment variable OPENAI_API_KEY and use OpenAI's embedding model "text-embedding-3-small" with the default dimension.+ The embedding function should take a list of text strings as input and output a list of embeddings.+ Defaults to None. By default, it will get OpenAI client by the environment variable OPENAI_API_KEY and use OpenAI's embedding model "text-embedding-3-small" with the default dimension.- `k (int, optional)`: The number of top passages to retrieve. Defaults to 3.### Methods@@ -36,10 +37,12 @@ MilvusRM(Search the Milvus collection for the top `k` passages matching the given query or queries, using embeddings generated via the default OpenAI embedding or the specified `embedding_function`.**Parameters:**+- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.**Returns:**+- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with schema `[{"id": str, "score": float, "long_text": str, "metadatas": dict }]`### Quickstart@@ -71,7 +74,6 @@ for result in results:print("Document:", result.long_text, "\n")```-#### Customized Embedding Function```pythondiff --git a/docs/api/retrieval_model_clients/MyScaleRM.md b/docs/api/retrieval_model_clients/MyScaleRM.mdindex d45144e1f..5fa8b87d1 100644--- a/docs/api/retrieval_model_clients/MyScaleRM.md+++ b/docs/api/retrieval_model_clients/MyScaleRM.md@@ -1,5 +1,5 @@----sidebar_position: 6+sidebar_position: 7---# retrieve.MyScaleRM@@ -9,6 +9,7 @@ sidebar_position: 6Initializes an instance of the `MyScaleRM` class, which is designed to use MyScaleDB (a ClickHouse fork optimized for vector similarity and full-text search) to retrieve documents based on query embeddings. This class supports embedding generation using either local models or OpenAI's API and manages database interactions efficiently.### Syntax+```pythonMyScaleRM(client: clickhouse_connect.driver.client.Client,@@ -22,7 +23,9 @@ MyScaleRM(local_embed_model: Optional[str] = None)```+## Parameters for `MyScaleRM` Constructor+- `client` (_clickhouse_connect.driver.client.Client_): A client connection to the MyScaleDB database, used to execute queries and manage interactions with the database.- `table` (_str_): Specifies the table within MyScaleDB from which data will be retrieved. This table should be equipped with a vector column for conducting similarity searches.- `database` (_str_, optional): The name of the database where the table is located, defaulting to `"default"`.@@ -34,21 +37,26 @@ MyScaleRM(- `local_embed_model` (_str, optional_): Specifies a local model for embedding generation, chosen if local computation is preferred.## Methods+### `forward`+Executes a retrieval operation based on a user's query and returns the top `k` relevant results using the embeddings generated by the specified method.### Syntax+```pythondef forward(self, user_query: str, k: Optional[int] = None) -> dspy.Prediction```## Parameters+- `user_query` (_str_): The query to retrieve matching passages.- `k` (_Optional[int], optional_): The number of top matches to retrieve. If not provided, it defaults to the `k` value set during class initialization.## Returns+- `dspy.Prediction`: Contains the retrieved passages, formatted as a list of `dotdict` objects. Each entry includes:- - **long_text (str)**: The text content of the retrieved passage.+ - **long_text (str)**: The text content of the retrieved passage.## Description@@ -77,4 +85,4 @@ passages = results.passagesfor passage in passages:print(passage['long_text'], "\n")-```\ No newline at end of file+```diff --git a/docs/api/retrieval_model_clients/Neo4jRM.md b/docs/api/retrieval_model_clients/Neo4jRM.mdindex ca153431d..ea7908aaa 100644--- a/docs/api/retrieval_model_clients/Neo4jRM.md+++ b/docs/api/retrieval_model_clients/Neo4jRM.md@@ -1,5 +1,5 @@----sidebar_position: 7+sidebar_position: 8---# retrieve.neo4j_rm@@ -34,6 +34,7 @@ You need to define the credentials as environment variables:- `OPENAI_API_KEY` (_str_): Specifies the API key required for authenticiating with OpenAI's services.**Parameters:**+- `index_name` (_str_): Specifies the name of the vector index to be used within Neo4j for organizing and querying data.- `text_node_property` (_str_, _optional_): Defines the specific property of nodes that will be returned.- `k` (_int_, _optional_): The number of top results to return from the retrieval operation. It defaults to 5 if not explicitly specified.@@ -41,7 +42,6 @@ You need to define the credentials as environment variables:- `embedding_provider` (_str_, _optional_): The name of the service provider for generating embeddings. Defaults to "openai" if not specified.- `embedding_model` (_str_, _optional_): The specific embedding model to use from the provider. By default, it uses the "text-embedding-ada-002" model from OpenAI.-### Methods#### `forward(self, query: [str], k: Optional[int] = None) -> dspy.Prediction`@@ -49,21 +49,23 @@ You need to define the credentials as environment variables:Search the neo4j vector index for the top `k` passages matching the given query or queries, using embeddings generated via the specified `embedding_model`.**Parameters:**-- `query` (str_): The query.++- `query` (str\_): The query.- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.**Returns:**+- `dspy.Prediction`: Contains the retrieved passages as a list of string with the prediction signature.ex:+```pythonPrediction(passages=['Passage 1 Lorem Ipsum awesome', 'Passage 2 Lorem Ipsum Youppidoo', 'Passage 3 Lorem Ipsum Yassssss'])```-### Quick Example how to use Neo4j in a local environment.-+### Quick Example how to use Neo4j in a local environment.```pythonfrom dspy.retrieve.neo4j_rm import Neo4jRMdiff --git a/docs/api/retrieval_model_clients/RAGatouilleRM.md b/docs/api/retrieval_model_clients/RAGatouilleRM.mdindex de7a73399..498047a29 100644--- a/docs/api/retrieval_model_clients/RAGatouilleRM.md+++ b/docs/api/retrieval_model_clients/RAGatouilleRM.md@@ -1,5 +1,5 @@----sidebar_position: 8+sidebar_position: 9---# retrieve.RAGatouilleRM@@ -13,12 +13,13 @@ class RAGatouilleRM(dspy.Retrieve):def __init__(self,index_root: str,- index_name: str,+ index_name: str,k: int = 3,):```**Parameters:**+- `index_root` (_str_): Folder path where your index is stored.- `index_name` (_str_): Name of the index you want to retrieve from.- `k` (_int_): The default number of passages to retrieve. Defaults to `3`.@@ -30,9 +31,10 @@ class RAGatouilleRM(dspy.Retrieve):Enables making queries to the RAGatouille-made index for retrieval. Internally, the method handles the specifics of preparing the query to obtain the response. The function handles the retrieval of the top-k passages based on the provided query.**Parameters:**+- `query_or_queries` (Union[str, List[str]]): Query string used for retrieval.- `k` (_int_, _optional_): Number of passages to retrieve. Defaults to 3.**Returns:**-- `dspy.Prediction`: List of k passages+- `dspy.Prediction`: List of k passagesdiff --git a/docs/api/retrieval_model_clients/SnowflakeRM.md b/docs/api/retrieval_model_clients/SnowflakeRM.mdindex c38619888..e94e98b32 100644--- a/docs/api/retrieval_model_clients/SnowflakeRM.md+++ b/docs/api/retrieval_model_clients/SnowflakeRM.md@@ -1,5 +1,5 @@----sidebar_position: 9+sidebar_position: 10---# retrieve.SnowflakeRMdiff --git a/docs/api/retrieval_model_clients/WatsonDiscovery.md b/docs/api/retrieval_model_clients/WatsonDiscovery.mdindex 3bb358ce8..198fbfdb8 100644--- a/docs/api/retrieval_model_clients/WatsonDiscovery.md+++ b/docs/api/retrieval_model_clients/WatsonDiscovery.md@@ -1,5 +1,5 @@----sidebar_position: 10+sidebar_position: 11---# retrieve.WatsonDiscoveryRM@@ -22,6 +22,7 @@ class WatsonDiscoveryRM:```**Parameters:**+- `apikey` (str): apikey for authentication purposes,- `url` (str): endpoint URL that includes the service instance ID- `version` (str): Release date of the version of the API you want to use. Specify dates in YYYY-MM-DD format.@@ -36,10 +37,12 @@ class WatsonDiscoveryRM:Search the Watson Discovery collection for the top `k` passages matching the given query or queries.**Parameters:**+- `query_or_queries` (_Union[str, list[str]]_): The query or list of queries to search for.- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.**Returns:**+- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with schema `[{"title":str, "long_text": str, "passage_score": float, "document_id": str, "collection_id": str, "start_offset": int, "end_offset": int, "field": str}]`### Quickstartdiff --git a/docs/api/retrieval_model_clients/YouRM.md b/docs/api/retrieval_model_clients/YouRM.mdindex df29b8a51..ac3f67746 100644--- a/docs/api/retrieval_model_clients/YouRM.md+++ b/docs/api/retrieval_model_clients/YouRM.md@@ -1,5 +1,5 @@----sidebar_position: 11+sidebar_position: 12---# retrieve.YouRM@@ -25,15 +25,15 @@ YouRM(**Parameters:**- `ydc_api_key` (Optional[str]): you.com API key, if `YDC_API_KEY` is not set in the environment-- `k` (int): If ``endpoint="search"``, the max snippets to return per search hit.- If ``endpoint="news"``, the max articles to return.+- `k` (int): If `endpoint="search"`, the max snippets to return per search hit.+ If `endpoint="news"`, the max articles to return.- `endpoint` (Literal["search", "news"]): you.com endpoints- `num_web_results` (Optional[int]): The max number of web results to return, must be under 20- `safesearch` (Optional[Literal["off", "moderate", "strict"]]): Safesearch settings, one of "off", "moderate", "strict", defaults to moderate- `country` (Optional[str]): Country code, ex: 'US' for United States, see API reference for more info- `search_lang` (Optional[str]): (News API) Language codes, ex: 'en' for English, see API reference for more info- `ui_lang` (Optional[str]): (News API) User interface language for the response, ex: 'en' for English.- See API reference for more info+ See API reference for more info- `spellcheck` (Optional[bool]): (News API) Whether to spell check query or not, defaults to True### Methods@@ -44,7 +44,6 @@ If `endpoint="search"`, search the web for the top `k` snippets matching the givIf `endpoint="news"`, search the web for the top `k` articles matching the given query or queries.-**Parameters:**- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.@@ -60,7 +59,6 @@ Obtain a You.com API key from [https://api.you.com/](https://api.you.com/).Export this key to an environment variable `YDC_API_KEY`.-```pythonfrom dspy.retrieve.you_rm import YouRMimport os