Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Updated markdown docs as well

Insop Song574f1a552f37
diff --git a/docs/language_models_client.md b/docs/language_models_client.md
index 9edb54980..f72d46cdf 100644
--- a/docs/language_models_client.md
+++ b/docs/language_models_client.md
@@ -23,6 +23,8 @@ for i, completion in enumerate(completions):
| Cohere | [Cohere Section](#cohere) |
| TGI | [TGI Section](#tgi) |
| VLLM | [VLLM Section](#vllm) |
+| Anyscale | [Anyscale Section](#anyscale) |
+| Together | [Together Section](#together) |
## OpenAI
@@ -155,4 +157,62 @@ Refer to [`dspy.TGI`](#tgi) documentation. Replace with `HFClientVLLM`.
### Methods
-Refer to [`dspy.OpenAI`](#openai) documentation.
\ No newline at end of file
+Refer to [`dspy.OpenAI`](#openai) documentation.
+
+## Anyscale
+
+### Usage
+
+```python
+lm = dspy.Anyscale(model="mistralai/Mistral-7B-Instruct-v0.1")
+```
+
+### Constructor
+
+The constructor initializes the base class `LM` and verifies the `api_key` for using Anyscale API.
+We expect the following environment variables to be set:
+- `ANYSCALE_API_KEY`: API key for Together.
+- `ANYSCALE_API_BASE`: API base URL for Together.
+
+
+```python
+class Anyscale(HFModel):
+ def __init__(self, model, **kwargs):
+```
+
+**Parameters:**
+- `model` (_str_): models hosted on Together.
+
+### Methods
+
+Refer to [`dspy.OpenAI`](#openai) documentation.
+
+
+## Together
+
+### Usage
+
+```python
+lm = dspy.Together(model="mistralai/Mistral-7B-v0.1")
+```
+
+### Constructor
+
+The constructor initializes the base class `LM` and verifies the `api_key` for using Together API.
+We expect the following environment variables to be set:
+- `TOGETHER_API_KEY`: API key for Together.
+- `TOGETHER_API_BASE`: API base URL for Together.
+
+
+```python
+class Together(HFModel):
+ def __init__(self, model, **kwargs):
+```
+
+**Parameters:**
+- `model` (_str_): models hosted on Together.
+- `stop` (_List[str]_, _optional_): List of stopping tokens to end generation.
+
+### Methods
+
+Refer to [`dspy.OpenAI`](#openai) documentation.
diff --git a/docs/using_local_models.md b/docs/using_local_models.md
index 66dc85bca..37c3391ab 100644
--- a/docs/using_local_models.md
+++ b/docs/using_local_models.md
@@ -10,6 +10,8 @@ DSPy supports various methods including `built-in wrappers`, `server integration
| Cohere | [Cohere Section](#cohere) |
| TGI | [TGI Section](#tgi) |
| VLLM | [VLLM Section](#vllm) |
+| MLC LLM | [MLC LLM Section](#mlc-llm) |
+| Ollama | [Ollama Section](#ollama) |
# HFModel
@@ -152,3 +154,45 @@ Please refer to the [official vLLM repository](https://github.com/vllm-project/v
llama = dspy.ChatModuleClient(model='dist/prebuilt/mlc-chat-Llama-2-7b-chat-hf-q4f16_1', model_path='dist/prebuilt/lib/Llama-2-7b-chat-hf-q4f16_1-cuda.so')
```
Please refer to the [official MLC repository](https://github.com/mlc-ai/mlc-llm) for more detailed information and [documentation](https://mlc.ai/mlc-llm/docs/get_started/try_out.html).
+
+# Ollama
+
+Ollama is a good software tool that allows you to run LLMs locally, such as Mistral, Llama2, and Phi.
+The following are the instructions to install and run Ollama.
+
+## Prerequisites
+
+Install Ollama by following the instructions from this page:
+
+- https://ollama.ai
+
+Download model: `ollama pull`
+
+Download a model by running the `ollama pull` command. You can download Mistral, Llama2, and Phi.
+
+```bash
+# download mistral
+ollama pull mistral
+```
+
+Here is the list of other models you can download:
+- https://ollama.ai/library
+
+## Running Ollama model
+
+Run model: `ollama run`
+
+You can test a model by running the model with the `ollama run` command.
+
+```bash
+# run mistral
+ollama run mistral
+```
+
+## Sending requests to the server
+
+Here is the code to load a model through Ollama:
+
+```python
+lm = dspy.OllamaLocal(model='mistral')
+```
\ No newline at end of file