Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

fix

dbczumarfc6fc6d6fc06
diff --git a/docs/api/retrieval_model_clients/AzureCognitiveSearch.md b/docs/api/retrieval_model_clients/AzureCognitiveSearch.md
index 832184563..bb210e830 100644
--- a/docs/api/retrieval_model_clients/AzureCognitiveSearch.md
+++ b/docs/api/retrieval_model_clients/AzureCognitiveSearch.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 2
+sidebar_position: 3
---
# retrieve.AzureCognitiveSearch
diff --git a/docs/api/retrieval_model_clients/ChromadbRM.md b/docs/api/retrieval_model_clients/ChromadbRM.md
index cfff233d2..97fbe26d4 100644
--- a/docs/api/retrieval_model_clients/ChromadbRM.md
+++ b/docs/api/retrieval_model_clients/ChromadbRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 3
+sidebar_position: 4
---
# retrieve.ChromadbRM
@@ -18,6 +18,7 @@ ChromadbRM(
```
**Parameters:**
+
- `collection_name` (_str_): The name of the chromadb collection.
- `persist_directory` (_str_): Path to the directory where chromadb data is persisted.
- `embedding_function` (_Optional[EmbeddingFunction[Embeddable]]_, _optional_): The function used for embedding documents and queries. Defaults to `DefaultEmbeddingFunction()` if not specified.
@@ -30,10 +31,12 @@ ChromadbRM(
Search the chromadb collection for the top `k` passages matching the given query or queries, using embeddings generated via the specified `embedding_function`.
**Parameters:**
+
- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.
- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.
**Returns:**
+
- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with schema `[{"id": str, "score": float, "long_text": str, "metadatas": dict }]`
### Quickstart with OpenAI Embeddings
diff --git a/docs/api/retrieval_model_clients/FaissRM.md b/docs/api/retrieval_model_clients/FaissRM.md
index 634904521..89aed38e7 100644
--- a/docs/api/retrieval_model_clients/FaissRM.md
+++ b/docs/api/retrieval_model_clients/FaissRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 4
+sidebar_position: 5
---
# retrieve.FaissRM
@@ -17,6 +17,7 @@ FaissRM(
```
**Parameters:**
+
- `document_chunks` (_List[str]_): a list of strings that comprises the corpus to search. You cannot add/insert/upsert to this list after creating this FaissRM object.
- `vectorizer` (_dsp.modules.sentence_vectorizer.BaseSentenceVectorizer_, _optional_): If not provided, a dsp.modules.sentence_vectorizer.SentenceTransformersVectorizer object is created and used.
- `k` (_int_, _optional_): The number of top passages to retrieve. Defaults to 3.
@@ -28,16 +29,17 @@ FaissRM(
Search the FaissRM vector database for the top `k` passages matching the given query or queries, using embeddings generated via the vectorizer specified at FaissRM construction time
**Parameters:**
+
- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.
**Returns:**
+
- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with a `long_text` attribute and an `index` attribute. The `index` attribute is the index in the document_chunks array provided to this FaissRM object at construction time.
### Quickstart with the default vectorizer
The **FaissRM** module provides a retriever that uses an in-memory Faiss vector database. This module does not include a vectorizer; instead it supports any subclass of **dsp.modules.sentence_vectorizer.BaseSentenceVectorizer**. If a vectorizer is not provided, an instance of **dsp.modules.sentence_vectorizer.SentenceTransformersVectorizer** is created and used by **FaissRM**. Note that the default embedding model for **SentenceTransformersVectorizer** is **all-MiniLM-L6-v2**
-
```python
import dspy
from dspy.retrieve.faiss_rm import FaissRM
diff --git a/docs/api/retrieval_model_clients/MilvusRM.md b/docs/api/retrieval_model_clients/MilvusRM.md
index e8fb41e42..2f10b77c5 100644
--- a/docs/api/retrieval_model_clients/MilvusRM.md
+++ b/docs/api/retrieval_model_clients/MilvusRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 5
+sidebar_position: 6
---
# retrieve.MilvusRM
@@ -20,13 +20,14 @@ MilvusRM(
```
**Parameters:**
+
- `collection_name (str)`: The name of the Milvus collection to query against.
- `uri (str, optional)`: The Milvus connection uri. Defaults to "http://localhost:19530".
- `token (str, optional)`: The Milvus connection token. Defaults to None.
- `db_name (str, optional)`: The Milvus database name. Defaults to "default".
- `embedding_function (callable, optional)`: The function to convert a list of text to embeddings.
- The embedding function should take a list of text strings as input and output a list of embeddings.
- Defaults to None. By default, it will get OpenAI client by the environment variable OPENAI_API_KEY and use OpenAI's embedding model "text-embedding-3-small" with the default dimension.
+ The embedding function should take a list of text strings as input and output a list of embeddings.
+ Defaults to None. By default, it will get OpenAI client by the environment variable OPENAI_API_KEY and use OpenAI's embedding model "text-embedding-3-small" with the default dimension.
- `k (int, optional)`: The number of top passages to retrieve. Defaults to 3.
### Methods
@@ -36,10 +37,12 @@ MilvusRM(
Search the Milvus collection for the top `k` passages matching the given query or queries, using embeddings generated via the default OpenAI embedding or the specified `embedding_function`.
**Parameters:**
+
- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.
- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.
**Returns:**
+
- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with schema `[{"id": str, "score": float, "long_text": str, "metadatas": dict }]`
### Quickstart
@@ -71,7 +74,6 @@ for result in results:
print("Document:", result.long_text, "\n")
```
-
#### Customized Embedding Function
```python
diff --git a/docs/api/retrieval_model_clients/MyScaleRM.md b/docs/api/retrieval_model_clients/MyScaleRM.md
index d45144e1f..5fa8b87d1 100644
--- a/docs/api/retrieval_model_clients/MyScaleRM.md
+++ b/docs/api/retrieval_model_clients/MyScaleRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 6
+sidebar_position: 7
---
# retrieve.MyScaleRM
@@ -9,6 +9,7 @@ sidebar_position: 6
Initializes an instance of the `MyScaleRM` class, which is designed to use MyScaleDB (a ClickHouse fork optimized for vector similarity and full-text search) to retrieve documents based on query embeddings. This class supports embedding generation using either local models or OpenAI's API and manages database interactions efficiently.
### Syntax
+
```python
MyScaleRM(
client: clickhouse_connect.driver.client.Client,
@@ -22,7 +23,9 @@ MyScaleRM(
local_embed_model: Optional[str] = None
)
```
+
## Parameters for `MyScaleRM` Constructor
+
- `client` (_clickhouse_connect.driver.client.Client_): A client connection to the MyScaleDB database, used to execute queries and manage interactions with the database.
- `table` (_str_): Specifies the table within MyScaleDB from which data will be retrieved. This table should be equipped with a vector column for conducting similarity searches.
- `database` (_str_, optional): The name of the database where the table is located, defaulting to `"default"`.
@@ -34,21 +37,26 @@ MyScaleRM(
- `local_embed_model` (_str, optional_): Specifies a local model for embedding generation, chosen if local computation is preferred.
## Methods
+
### `forward`
+
Executes a retrieval operation based on a user's query and returns the top `k` relevant results using the embeddings generated by the specified method.
### Syntax
+
```python
def forward(self, user_query: str, k: Optional[int] = None) -> dspy.Prediction
```
## Parameters
+
- `user_query` (_str_): The query to retrieve matching passages.
- `k` (_Optional[int], optional_): The number of top matches to retrieve. If not provided, it defaults to the `k` value set during class initialization.
## Returns
+
- `dspy.Prediction`: Contains the retrieved passages, formatted as a list of `dotdict` objects. Each entry includes:
- - **long_text (str)**: The text content of the retrieved passage.
+ - **long_text (str)**: The text content of the retrieved passage.
## Description
@@ -77,4 +85,4 @@ passages = results.passages
for passage in passages:
print(passage['long_text'], "\n")
-```
\ No newline at end of file
+```
diff --git a/docs/api/retrieval_model_clients/Neo4jRM.md b/docs/api/retrieval_model_clients/Neo4jRM.md
index ca153431d..ea7908aaa 100644
--- a/docs/api/retrieval_model_clients/Neo4jRM.md
+++ b/docs/api/retrieval_model_clients/Neo4jRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 7
+sidebar_position: 8
---
# retrieve.neo4j_rm
@@ -34,6 +34,7 @@ You need to define the credentials as environment variables:
- `OPENAI_API_KEY` (_str_): Specifies the API key required for authenticiating with OpenAI's services.
**Parameters:**
+
- `index_name` (_str_): Specifies the name of the vector index to be used within Neo4j for organizing and querying data.
- `text_node_property` (_str_, _optional_): Defines the specific property of nodes that will be returned.
- `k` (_int_, _optional_): The number of top results to return from the retrieval operation. It defaults to 5 if not explicitly specified.
@@ -41,7 +42,6 @@ You need to define the credentials as environment variables:
- `embedding_provider` (_str_, _optional_): The name of the service provider for generating embeddings. Defaults to "openai" if not specified.
- `embedding_model` (_str_, _optional_): The specific embedding model to use from the provider. By default, it uses the "text-embedding-ada-002" model from OpenAI.
-
### Methods
#### `forward(self, query: [str], k: Optional[int] = None) -> dspy.Prediction`
@@ -49,21 +49,23 @@ You need to define the credentials as environment variables:
Search the neo4j vector index for the top `k` passages matching the given query or queries, using embeddings generated via the specified `embedding_model`.
**Parameters:**
-- `query` (str_): The query.
+
+- `query` (str\_): The query.
- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.
**Returns:**
+
- `dspy.Prediction`: Contains the retrieved passages as a list of string with the prediction signature.
ex:
+
```python
Prediction(
passages=['Passage 1 Lorem Ipsum awesome', 'Passage 2 Lorem Ipsum Youppidoo', 'Passage 3 Lorem Ipsum Yassssss']
)
```
-### Quick Example how to use Neo4j in a local environment.
-
+### Quick Example how to use Neo4j in a local environment.
```python
from dspy.retrieve.neo4j_rm import Neo4jRM
diff --git a/docs/api/retrieval_model_clients/RAGatouilleRM.md b/docs/api/retrieval_model_clients/RAGatouilleRM.md
index de7a73399..498047a29 100644
--- a/docs/api/retrieval_model_clients/RAGatouilleRM.md
+++ b/docs/api/retrieval_model_clients/RAGatouilleRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 8
+sidebar_position: 9
---
# retrieve.RAGatouilleRM
@@ -13,12 +13,13 @@ class RAGatouilleRM(dspy.Retrieve):
def __init__(
self,
index_root: str,
- index_name: str,
+ index_name: str,
k: int = 3,
):
```
**Parameters:**
+
- `index_root` (_str_): Folder path where your index is stored.
- `index_name` (_str_): Name of the index you want to retrieve from.
- `k` (_int_): The default number of passages to retrieve. Defaults to `3`.
@@ -30,9 +31,10 @@ class RAGatouilleRM(dspy.Retrieve):
Enables making queries to the RAGatouille-made index for retrieval. Internally, the method handles the specifics of preparing the query to obtain the response. The function handles the retrieval of the top-k passages based on the provided query.
**Parameters:**
+
- `query_or_queries` (Union[str, List[str]]): Query string used for retrieval.
- `k` (_int_, _optional_): Number of passages to retrieve. Defaults to 3.
**Returns:**
-- `dspy.Prediction`: List of k passages
+- `dspy.Prediction`: List of k passages
diff --git a/docs/api/retrieval_model_clients/SnowflakeRM.md b/docs/api/retrieval_model_clients/SnowflakeRM.md
index c38619888..e94e98b32 100644
--- a/docs/api/retrieval_model_clients/SnowflakeRM.md
+++ b/docs/api/retrieval_model_clients/SnowflakeRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 9
+sidebar_position: 10
---
# retrieve.SnowflakeRM
diff --git a/docs/api/retrieval_model_clients/WatsonDiscovery.md b/docs/api/retrieval_model_clients/WatsonDiscovery.md
index 3bb358ce8..198fbfdb8 100644
--- a/docs/api/retrieval_model_clients/WatsonDiscovery.md
+++ b/docs/api/retrieval_model_clients/WatsonDiscovery.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 10
+sidebar_position: 11
---
# retrieve.WatsonDiscoveryRM
@@ -22,6 +22,7 @@ class WatsonDiscoveryRM:
```
**Parameters:**
+
- `apikey` (str): apikey for authentication purposes,
- `url` (str): endpoint URL that includes the service instance ID
- `version` (str): Release date of the version of the API you want to use. Specify dates in YYYY-MM-DD format.
@@ -36,10 +37,12 @@ class WatsonDiscoveryRM:
Search the Watson Discovery collection for the top `k` passages matching the given query or queries.
**Parameters:**
+
- `query_or_queries` (_Union[str, list[str]]_): The query or list of queries to search for.
- `k` (_Optional[int]_, _optional_): The number of results to retrieve. If not specified, defaults to the value set during initialization.
**Returns:**
+
- `dspy.Prediction`: Contains the retrieved passages, each represented as a `dotdict` with schema `[{"title":str, "long_text": str, "passage_score": float, "document_id": str, "collection_id": str, "start_offset": int, "end_offset": int, "field": str}]`
### Quickstart
diff --git a/docs/api/retrieval_model_clients/YouRM.md b/docs/api/retrieval_model_clients/YouRM.md
index df29b8a51..ac3f67746 100644
--- a/docs/api/retrieval_model_clients/YouRM.md
+++ b/docs/api/retrieval_model_clients/YouRM.md
@@ -1,5 +1,5 @@
---
-sidebar_position: 11
+sidebar_position: 12
---
# retrieve.YouRM
@@ -25,15 +25,15 @@ YouRM(
**Parameters:**
- `ydc_api_key` (Optional[str]): you.com API key, if `YDC_API_KEY` is not set in the environment
-- `k` (int): If ``endpoint="search"``, the max snippets to return per search hit.
- If ``endpoint="news"``, the max articles to return.
+- `k` (int): If `endpoint="search"`, the max snippets to return per search hit.
+ If `endpoint="news"`, the max articles to return.
- `endpoint` (Literal["search", "news"]): you.com endpoints
- `num_web_results` (Optional[int]): The max number of web results to return, must be under 20
- `safesearch` (Optional[Literal["off", "moderate", "strict"]]): Safesearch settings, one of "off", "moderate", "strict", defaults to moderate
- `country` (Optional[str]): Country code, ex: 'US' for United States, see API reference for more info
- `search_lang` (Optional[str]): (News API) Language codes, ex: 'en' for English, see API reference for more info
- `ui_lang` (Optional[str]): (News API) User interface language for the response, ex: 'en' for English.
- See API reference for more info
+ See API reference for more info
- `spellcheck` (Optional[bool]): (News API) Whether to spell check query or not, defaults to True
### Methods
@@ -44,7 +44,6 @@ If `endpoint="search"`, search the web for the top `k` snippets matching the giv
If `endpoint="news"`, search the web for the top `k` articles matching the given query or queries.
-
**Parameters:**
- `query_or_queries` (_Union[str, List[str]]_): The query or list of queries to search for.
@@ -60,7 +59,6 @@ Obtain a You.com API key from [https://api.you.com/](https://api.you.com/).
Export this key to an environment variable `YDC_API_KEY`.
-
```python
from dspy.retrieve.you_rm import YouRM
import os