DSPy: The framework for programming—not prompting—language models
Python
Updated markdown docs as well
Insop Song574f1a552f37
diff --git a/docs/language_models_client.md b/docs/language_models_client.mdindex 9edb54980..f72d46cdf 100644--- a/docs/language_models_client.md+++ b/docs/language_models_client.md@@ -23,6 +23,8 @@ for i, completion in enumerate(completions):| Cohere | [Cohere Section](#cohere) || TGI | [TGI Section](#tgi) || VLLM | [VLLM Section](#vllm) |+| Anyscale | [Anyscale Section](#anyscale) |+| Together | [Together Section](#together) |## OpenAI@@ -155,4 +157,62 @@ Refer to [`dspy.TGI`](#tgi) documentation. Replace with `HFClientVLLM`.### Methods-Refer to [`dspy.OpenAI`](#openai) documentation.\ No newline at end of file+Refer to [`dspy.OpenAI`](#openai) documentation.++## Anyscale++### Usage++```python+lm = dspy.Anyscale(model="mistralai/Mistral-7B-Instruct-v0.1")+```++### Constructor++The constructor initializes the base class `LM` and verifies the `api_key` for using Anyscale API.+We expect the following environment variables to be set:+- `ANYSCALE_API_KEY`: API key for Together.+- `ANYSCALE_API_BASE`: API base URL for Together.+++```python+class Anyscale(HFModel):+ def __init__(self, model, **kwargs):+```++**Parameters:**+- `model` (_str_): models hosted on Together.++### Methods++Refer to [`dspy.OpenAI`](#openai) documentation.+++## Together++### Usage++```python+lm = dspy.Together(model="mistralai/Mistral-7B-v0.1")+```++### Constructor++The constructor initializes the base class `LM` and verifies the `api_key` for using Together API.+We expect the following environment variables to be set:+- `TOGETHER_API_KEY`: API key for Together.+- `TOGETHER_API_BASE`: API base URL for Together.+++```python+class Together(HFModel):+ def __init__(self, model, **kwargs):+```++**Parameters:**+- `model` (_str_): models hosted on Together.+- `stop` (_List[str]_, _optional_): List of stopping tokens to end generation.++### Methods++Refer to [`dspy.OpenAI`](#openai) documentation.diff --git a/docs/using_local_models.md b/docs/using_local_models.mdindex 66dc85bca..37c3391ab 100644--- a/docs/using_local_models.md+++ b/docs/using_local_models.md@@ -10,6 +10,8 @@ DSPy supports various methods including `built-in wrappers`, `server integration| Cohere | [Cohere Section](#cohere) || TGI | [TGI Section](#tgi) || VLLM | [VLLM Section](#vllm) |+| MLC LLM | [MLC LLM Section](#mlc-llm) |+| Ollama | [Ollama Section](#ollama) |# HFModel@@ -152,3 +154,45 @@ Please refer to the [official vLLM repository](https://github.com/vllm-project/vllama = dspy.ChatModuleClient(model='dist/prebuilt/mlc-chat-Llama-2-7b-chat-hf-q4f16_1', model_path='dist/prebuilt/lib/Llama-2-7b-chat-hf-q4f16_1-cuda.so')```Please refer to the [official MLC repository](https://github.com/mlc-ai/mlc-llm) for more detailed information and [documentation](https://mlc.ai/mlc-llm/docs/get_started/try_out.html).++# Ollama++Ollama is a good software tool that allows you to run LLMs locally, such as Mistral, Llama2, and Phi.+The following are the instructions to install and run Ollama.++## Prerequisites++Install Ollama by following the instructions from this page:++- https://ollama.ai++Download model: `ollama pull`++Download a model by running the `ollama pull` command. You can download Mistral, Llama2, and Phi.++```bash+# download mistral+ollama pull mistral+```++Here is the list of other models you can download:+- https://ollama.ai/library++## Running Ollama model++Run model: `ollama run`++You can test a model by running the model with the `ollama run` command.++```bash+# run mistral+ollama run mistral+```++## Sending requests to the server++Here is the code to load a model through Ollama:++```python+lm = dspy.OllamaLocal(model='mistral')+```\ No newline at end of file