Skip to content

The agent engineering platform.

community: Rename deployments server to AI gateway (#26368)

Harutaka Kawamura6ed50e78c96d
diff --git a/docs/docs/integrations/providers/mlflow.mdx b/docs/docs/integrations/providers/mlflow.mdx
index cb4d5aba84..861154a0b8 100644
--- a/docs/docs/integrations/providers/mlflow.mdx
+++ b/docs/docs/integrations/providers/mlflow.mdx
@@ -1,12 +1,12 @@
-# MLflow Deployments for LLMs
+# MLflow AI Gateway for LLMs
->[The MLflow Deployments for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large
+>[The MLflow AI Gateway for LLMs](https://www.mlflow.org/docs/latest/llms/deployments/index.html) is a powerful tool designed to streamline the usage and management of various large
> language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface
> that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests.
## Installation and Setup
-Install `mlflow` with MLflow Deployments dependencies:
+Install `mlflow` with MLflow GenAI dependencies:
```sh
pip install 'mlflow[genai]'
@@ -39,10 +39,10 @@ endpoints:
openai_api_key: $OPENAI_API_KEY
```
-Start the deployments server:
+Start the gateway server:
```sh
-mlflow deployments start-server --config-path /path/to/config.yaml
+mlflow gateway start --config-path /path/to/config.yaml
```
## Example provided by `MLflow`
diff --git a/docs/docs/integrations/providers/mlflow_ai_gateway.mdx b/docs/docs/integrations/providers/mlflow_ai_gateway.mdx
deleted file mode 100644
index 912ea449eb..0000000000
--- a/docs/docs/integrations/providers/mlflow_ai_gateway.mdx
+++ /dev/null
@@ -1,160 +0,0 @@
-# MLflow AI Gateway
-
-:::warning
-
-MLflow AI Gateway has been deprecated. Please use [MLflow Deployments for LLMs](/docs/integrations/providers/mlflow/) instead.
-
-:::
-
->[The MLflow AI Gateway](https://www.mlflow.org/docs/latest/index.html) service is a powerful tool designed to streamline the usage and management of various large
-> language model (LLM) providers, such as OpenAI and Anthropic, within an organization. It offers a high-level interface
-> that simplifies the interaction with these services by providing a unified endpoint to handle specific LLM related requests.
-
-## Installation and Setup
-
-Install `mlflow` with MLflow AI Gateway dependencies:
-
-```sh
-pip install 'mlflow[gateway]'
-```
-
-Set the OpenAI API key as an environment variable:
-
-```sh
-export OPENAI_API_KEY=...
-```
-
-Create a configuration file:
-
-```yaml
-routes:
- - name: completions
- route_type: llm/v1/completions
- model:
- provider: openai
- name: text-davinci-003
- config:
- openai_api_key: $OPENAI_API_KEY
-
- - name: embeddings
- route_type: llm/v1/embeddings
- model:
- provider: openai
- name: text-embedding-ada-002
- config:
- openai_api_key: $OPENAI_API_KEY
-```
-
-Start the Gateway server:
-
-```sh
-mlflow gateway start --config-path /path/to/config.yaml
-```
-
-## Example provided by `MLflow`
-
->The `mlflow.langchain` module provides an API for logging and loading `LangChain` models.
-> This module exports multivariate LangChain models in the langchain flavor and univariate LangChain
-> models in the pyfunc flavor.
-
-See the [API documentation and examples](https://www.mlflow.org/docs/latest/python_api/mlflow.langchain.html?highlight=langchain#module-mlflow.langchain).
-
-
-
-## Completions Example
-
-```python
-import mlflow
-from langchain.chains import LLMChain, PromptTemplate
-from langchain_community.llms import MlflowAIGateway
-
-gateway = MlflowAIGateway(
- gateway_uri="http://127.0.0.1:5000",
- route="completions",
- params={
- "temperature": 0.0,
- "top_p": 0.1,
- },
-)
-
-llm_chain = LLMChain(
- llm=gateway,
- prompt=PromptTemplate(
- input_variables=["adjective"],
- template="Tell me a {adjective} joke",
- ),
-)
-result = llm_chain.run(adjective="funny")
-print(result)
-
-with mlflow.start_run():
- model_info = mlflow.langchain.log_model(chain, "model")
-
-model = mlflow.pyfunc.load_model(model_info.model_uri)
-print(model.predict([{"adjective": "funny"}]))
-```
-
-## Embeddings Example
-
-```python
-from langchain_community.embeddings import MlflowAIGatewayEmbeddings
-
-embeddings = MlflowAIGatewayEmbeddings(
- gateway_uri="http://127.0.0.1:5000",
- route="embeddings",
-)
-
-print(embeddings.embed_query("hello"))
-print(embeddings.embed_documents(["hello"]))
-```
-
-## Chat Example
-
-```python
-from langchain_community.chat_models import ChatMLflowAIGateway
-from langchain_core.messages import HumanMessage, SystemMessage
-
-chat = ChatMLflowAIGateway(
- gateway_uri="http://127.0.0.1:5000",
- route="chat",
- params={
- "temperature": 0.1
- }
-)
-
-messages = [
- SystemMessage(
- content="You are a helpful assistant that translates English to French."
- ),
- HumanMessage(
- content="Translate this sentence from English to French: I love programming."
- ),
-]
-print(chat(messages))
-```
-
-## Databricks MLflow AI Gateway
-
-Databricks MLflow AI Gateway is in private preview.
-Please contact a Databricks representative to enroll in the preview.
-
-```python
-from langchain.chains import LLMChain
-from langchain_core.prompts import PromptTemplate
-from langchain_community.llms import MlflowAIGateway
-
-gateway = MlflowAIGateway(
- gateway_uri="databricks",
- route="completions",
-)
-
-llm_chain = LLMChain(
- llm=gateway,
- prompt=PromptTemplate(
- input_variables=["adjective"],
- template="Tell me a {adjective} joke",
- ),
-)
-result = llm_chain.run(adjective="funny")
-print(result)
-```
diff --git a/docs/vercel.json b/docs/vercel.json
index 5f88f694cd..ed15a82a06 100644
--- a/docs/vercel.json
+++ b/docs/vercel.json
@@ -73,6 +73,10 @@
{
"source": "/v0.2/docs/templates/:path(.*/?)*",
"destination": "https://github.com/langchain-ai/langchain/tree/master/templates/:path*"
+ },
+ {
+ "source": "/docs/integrations/providers/mlflow_ai_gateway(/?)",
+ "destination": "/docs/integrations/providers/mlflow/"
}
]
}