Skip to content

The agent engineering platform.

cli/docs: llm integration template standardization (#24795)

Erick Friiscdaea17b3e95
diff --git a/libs/cli/langchain_cli/integration_template/docs/chat.ipynb b/libs/cli/langchain_cli/integration_template/docs/chat.ipynb
index e19681176f..2b838e3b09 100644
--- a/libs/cli/langchain_cli/integration_template/docs/chat.ipynb
+++ b/libs/cli/langchain_cli/integration_template/docs/chat.ipynb
@@ -62,7 +62,8 @@
"import getpass\n",
"import os\n",
"\n",
- "os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")"
+ "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n",
+ " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")"
]
},
{
@@ -80,8 +81,8 @@
"metadata": {},
"outputs": [],
"source": [
- "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
- "# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
+ "# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
+ "# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")"
]
},
{
@@ -196,7 +197,7 @@
"source": [
"from langchain_core.prompts import ChatPromptTemplate\n",
"\n",
- "prompt = ChatPromptTemplate.from_messages(\n",
+ "prompt = ChatPromptTemplate(\n",
" [\n",
" (\n",
" \"system\",\n",
diff --git a/libs/cli/langchain_cli/integration_template/docs/llms.ipynb b/libs/cli/langchain_cli/integration_template/docs/llms.ipynb
index 69c34d1af0..2629c7718d 100644
--- a/libs/cli/langchain_cli/integration_template/docs/llms.ipynb
+++ b/libs/cli/langchain_cli/integration_template/docs/llms.ipynb
@@ -17,9 +17,75 @@
"source": [
"# __ModuleName__LLM\n",
"\n",
- "This example goes over how to use LangChain to interact with `__ModuleName__` models.\n",
+ "- [ ] TODO: Make sure API reference link is correct\n",
"\n",
- "## Installation"
+ "This will help you get started with __ModuleName__ completion models (LLMs) using LangChain. For detailed documentation on `__ModuleName__LLM` features and configuration options, please refer to the [API reference](https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html).\n",
+ "\n",
+ "## Overview\n",
+ "### Integration details\n",
+ "\n",
+ "- TODO: Fill in table features.\n",
+ "- TODO: Remove JS support link if not relevant, otherwise ensure link is correct.\n",
+ "- TODO: Make sure API reference links are correct.\n",
+ "\n",
+ "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/__package_name_short_snake__) | Package downloads | Package latest |\n",
+ "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
+ "| [__ModuleName__LLM](https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html) | [__package_name__](https://api.python.langchain.com/en/latest/__package_name_short_snake___api_reference.html) | ✅/❌ | beta/❌ | ✅/❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/__package_name__?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/__package_name__?style=flat-square&label=%20) |\n",
+ "\n",
+ "## Setup\n",
+ "\n",
+ "- [ ] TODO: Update with relevant info.\n",
+ "\n",
+ "To access __ModuleName__ models you'll need to create a/an __ModuleName__ account, get an API key, and install the `__package_name__` integration package.\n",
+ "\n",
+ "### Credentials\n",
+ "\n",
+ "- TODO: Update with relevant info.\n",
+ "\n",
+ "Head to (TODO: link) to sign up to __ModuleName__ and generate an API key. Once you've done this set the __MODULE_NAME___API_KEY environment variable:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "bc51e756",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import getpass\n",
+ "import os\n",
+ "\n",
+ "if not os.getenv(\"__MODULE_NAME___API_KEY\"):\n",
+ " os.environ[\"__MODULE_NAME___API_KEY\"] = getpass.getpass(\"Enter your __ModuleName__ API key: \")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4b6e1ca6",
+ "metadata": {},
+ "source": [
+ "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "196c2b41",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
+ "# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "809c6577",
+ "metadata": {},
+ "source": [
+ "### Installation\n",
+ "\n",
+ "The LangChain __ModuleName__ integration lives in the `__package_name__` package:"
]
},
{
@@ -29,22 +95,48 @@
"metadata": {},
"outputs": [],
"source": [
- "# install package\n",
- "!pip install -U __package_name__"
+ "%pip install -qU __package_name__"
]
},
{
"cell_type": "markdown",
- "id": "0ee90032",
+ "id": "0a760037",
"metadata": {},
"source": [
- "## Environment Setup\n",
+ "## Instantiation\n",
"\n",
- "Make sure to set the following environment variables:\n",
+ "Now we can instantiate our model object and generate chat completions:\n",
"\n",
- "- TODO: fill out relevant environment variables or secrets\n",
+ "- TODO: Update model instantiation with relevant params."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a0562a13",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from __module_name__ import __ModuleName__LLM\n",
"\n",
- "## Usage"
+ "llm = __ModuleName__LLM(\n",
+ " model=\"model-name\",\n",
+ " temperature=0,\n",
+ " max_tokens=None,\n",
+ " timeout=None,\n",
+ " max_retries=2,\n",
+ " # other params...\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0ee90032",
+ "metadata": {},
+ "source": [
+ "## Invocation\n",
+ "\n",
+ "- [ ] TODO: Run cells so output can be seen."
]
},
{
@@ -56,20 +148,64 @@
},
"outputs": [],
"source": [
- "from langchain_core.prompts import PromptTemplate\n",
- "from __module_name__.llms import __ModuleName__LLM\n",
+ "input_text = \"__ModuleName__ is an AI company that \"\n",
"\n",
- "template = \"\"\"Question: {question}\n",
+ "completion = llm.invoke(input_text)\n",
+ "completion"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "add38532",
+ "metadata": {},
+ "source": [
+ "## Chaining\n",
"\n",
- "Answer: Let's think step by step.\"\"\"\n",
+ "We can [chain](/docs/how_to/sequence/) our completion model with a prompt template like so:\n",
"\n",
- "prompt = PromptTemplate.from_string(template)\n",
+ "- TODO: Run cells so output can be seen."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "078e9db2",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from langchain_core.prompts import ChatPromptTemplate\n",
+ "\n",
+ "prompt = PromptTemplate(\n",
+ " \"How to say {input} in {output_language}:\\n\"\n",
+ ")\n",
"\n",
- "model = __ModuleName__LLM()\n",
+ "chain = prompt | llm\n",
+ "chain.invoke(\n",
+ " {\n",
+ " \"output_language\": \"German\",\n",
+ " \"input\": \"I love programming.\",\n",
+ " }\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e99eef30",
+ "metadata": {},
+ "source": [
+ "## TODO: Any functionality specific to this model provider\n",
"\n",
- "chain = prompt | model\n",
+ "E.g. creating/using finetuned models via this provider. Delete if not relevant"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e9bdfcef",
+ "metadata": {},
+ "source": [
+ "## API reference\n",
"\n",
- "chain.invoke({\"question\": \"What is LangChain?\"})"
+ "For detailed documentation of all `__ModuleName__LLM` features and configurations head to the API reference: https://api.python.langchain.com/en/latest/llms/__module_name__.llms.__ModuleName__LLM.html"
]
}
],
diff --git a/libs/cli/langchain_cli/integration_template/integration_template/llms.py b/libs/cli/langchain_cli/integration_template/integration_template/llms.py
index 675e95f158..2dbe4ac918 100644
--- a/libs/cli/langchain_cli/integration_template/integration_template/llms.py
+++ b/libs/cli/langchain_cli/integration_template/integration_template/llms.py
@@ -2,36 +2,110 @@
from typing import (
Any,
- AsyncIterator,
- Iterator,
List,
Optional,
)
from langchain_core.callbacks import (
- AsyncCallbackManagerForLLMRun,
CallbackManagerForLLMRun,
)
from langchain_core.language_models import BaseLLM
-from langchain_core.outputs import GenerationChunk, LLMResult
+from langchain_core.outputs import LLMResult
class __ModuleName__LLM(BaseLLM):
- """__ModuleName__LLM large language models.
+ """__ModuleName__ completion model integration.
- Example:
+ # TODO: Replace with relevant packages, env vars.
+ Setup:
+ Install ``__package_name__`` and set environment variable ``__MODULE_NAME___API_KEY``.
+
+ .. code-block:: bash
+
+ pip install -U __package_name__
+ export __MODULE_NAME___API_KEY="your-api-key"
+
+ # TODO: Populate with relevant params.
+ Key init args — completion params:
+ model: str
+ Name of __ModuleName__ model to use.
+ temperature: float
+ Sampling temperature.
+ max_tokens: Optional[int]
+ Max number of tokens to generate.
+
+ # TODO: Populate with relevant params.
+ Key init args — client params:
+ timeout: Optional[float]
+ Timeout for requests.
+ max_retries: int
+ Max number of retries.
+ api_key: Optional[str]
+ __ModuleName__ API key. If not passed in will be read from env var __MODULE_NAME___API_KEY.
+
+ See full list of supported init args and their descriptions in the params section.
+
+ # TODO: Replace with relevant init params.
+ Instantiate:
.. code-block:: python
from __module_name__ import __ModuleName__LLM
- model = __ModuleName__LLM()
- model.invoke("Come up with 10 names for a song about parrots")
- """
+ llm = __ModuleName__LLM(
+ model="...",
+ temperature=0,
+ max_tokens=None,
+ timeout=None,
+ max_retries=2,
+ # api_key="...",
+ # other params...
+ )
- @property
- def _llm_type(self) -> str:
- """Return type of LLM."""
- return "__package_name_short__-llm"
+ Invoke:
+ .. code-block:: python
+
+ input_text = "The meaning of life is "
+ llm.invoke(input_text)
+
+ .. code-block:: python
+
+ # TODO: Example output.
+
+ # TODO: Delete if token-level streaming isn't supported.
+ Stream:
+ .. code-block:: python
+
+ for chunk in llm.stream(input_text):
+ print(chunk)
+
+ .. code-block:: python
+
+ # TODO: Example output.
+
+ .. code-block:: python
+
+ ''.join(llm.stream(input_text))
+
+ .. code-block:: python
+
+ # TODO: Example output.
+
+ # TODO: Delete if native async isn't supported.
+ Async:
+ .. code-block:: python
+
+ await llm.ainvoke(input_text)
+
+ # stream:
+ # async for chunk in (await llm.astream(input_text))
+
+ # batch:
+ # await llm.abatch([input_text])
+
+ .. code-block:: python
+
+ # TODO: Example output.
+ """
# TODO: This method must be implemented to generate text completions.
def _generate(
@@ -45,32 +119,37 @@ class __ModuleName__LLM(BaseLLM):
# TODO: Implement if __ModuleName__LLM supports async generation. Otherwise
# delete method.
- async def _agenerate(
- self,
- prompts: List[str],
- stop: Optional[List[str]] = None,
- run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
- **kwargs: Any,
- ) -> LLMResult:
- raise NotImplementedError
+ # async def _agenerate(
+ # self,
+ # prompts: List[str],
+ # stop: Optional[List[str]] = None,
+ # run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
+ # **kwargs: Any,
+ # ) -> LLMResult:
+ # raise NotImplementedError
# TODO: Implement if __ModuleName__LLM supports streaming. Otherwise delete method.
- def _stream(
- self,
- prompt: str,
- stop: Optional[List[str]] = None,
- run_manager: Optional[CallbackManagerForLLMRun] = None,
- **kwargs: Any,
- ) -> Iterator[GenerationChunk]:
- raise NotImplementedError
+ # def _stream(
+ # self,
+ # prompt: str,
+ # stop: Optional[List[str]] = None,
+ # run_manager: Optional[CallbackManagerForLLMRun] = None,
+ # **kwargs: Any,
+ # ) -> Iterator[GenerationChunk]:
+ # raise NotImplementedError
# TODO: Implement if __ModuleName__LLM supports async streaming. Otherwise delete
# method.
- async def _astream(
- self,
- prompt: str,
- stop: Optional[List[str]] = None,
- run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
- **kwargs: Any,
- ) -> AsyncIterator[GenerationChunk]:
- raise NotImplementedError
+ # async def _astream(
+ # self,
+ # prompt: str,
+ # stop: Optional[List[str]] = None,
+ # run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
+ # **kwargs: Any,
+ # ) -> AsyncIterator[GenerationChunk]:
+ # raise NotImplementedError
+
+ @property
+ def _llm_type(self) -> str:
+ """Return type of LLM."""
+ return "__package_name_short__-llm"