The agent engineering platform.
Python agentsaiai-agentsanthropicchatgptdeepagentsenterpriseframeworkgeminigenerative-ailangchainlanggraphllmmultiagentopen-sourceopenaipydanticpythonragtypescript
community: add Featherless.ai integration (#31250)
DarinVerheijke49fbcec34f65
diff --git a/docs/docs/integrations/chat/featherless_ai.ipynb b/docs/docs/integrations/chat/featherless_ai.ipynbnew file mode 100644index 0000000000..f8b18322d2--- /dev/null+++ b/docs/docs/integrations/chat/featherless_ai.ipynb@@ -0,0 +1,308 @@+{+ "cells": [+ {+ "cell_type": "raw",+ "id": "afaf8039",+ "metadata": {+ "vscode": {+ "languageId": "raw"+ }+ },+ "source": [+ "---\n",+ "sidebar_label: Featherless AI\n",+ "---"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "e49f1e0d",+ "metadata": {},+ "source": [+ "# ChatFeatherlessAi\n",+ "\n",+ "\n",+ "This will help you getting started with FeatherlessAi [chat models](/docs/concepts/chat_models). For detailed documentation of all ChatFeatherlessAi features and configurations head to the [API reference](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.ChatFeatherlessAi.html).\n",+ "\n",+ "- See https://featherless.ai/ for an example.\n",+ "\n",+ "## Overview\n",+ "### Integration details\n",+ "\n",+ "\n",+ "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/docs/integrations/chat/__package_name_short_snake__) | Package downloads | Package latest |\n",+ "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",+ "| [ChatFeatherlessAi](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/__module_name__.chat_models.ChatFeatherlessAi.html) | [langchain-featherless-ai](https://python.langchain.com/api_reference/__package_name_short_snake__/) | ✅ | ❌ | ❌ |  |  |\n",+ "\n",+ "### Model features\n",+ "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",+ "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",+ "| ❌ | ❌ | ✅| ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | \n",+ "\n",+ "## Setup\n",+ "\n",+ "\n",+ "To access Featherless AI models you'll need to create a/an Featherless AI account, get an API key, and install the `langchain-featherless-ai` integration package.\n",+ "\n",+ "### Credentials\n",+ "\n",+ "\n",+ "Head to https://featherless.ai/ to sign up to FeatherlessAI and generate an API key. Once you've done this set the FEATHERLESSAI_API_KEY environment variable:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 26,+ "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94",+ "metadata": {},+ "outputs": [],+ "source": [+ "import getpass\n",+ "import os\n",+ "\n",+ "if not os.getenv(\"FEATHERLESSAI_API_KEY\"):\n",+ " os.environ[\"FEATHERLESSAI_API_KEY\"] = getpass.getpass(\n",+ " \"Enter your FeatherlessAI API key: \"\n",+ " )"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "72ee0c4b-9764-423a-9dbf-95129e185210",+ "metadata": {},+ "source": [+ "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 2,+ "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de",+ "metadata": {},+ "outputs": [],+ "source": [+ "# os.environ[\"LANGSMITH_TRACING\"] = \"true\"\n",+ "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "0730d6a1-c893-4840-9817-5e5251676d5d",+ "metadata": {},+ "source": [+ "### Installation\n",+ "\n",+ "The LangChain FeatherlessAi integration lives in the `langchain-featherless-ai` package:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "652d6238-1f87-422a-b135-f5abbb8652fc",+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "Note: you may need to restart the kernel to use updated packages.\n"+ ]+ }+ ],+ "source": [+ "%pip install -qU langchain-featherless-ai"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "a38cde65-254d-4219-a441-068766c0d4b5",+ "metadata": {},+ "source": [+ "## Instantiation\n",+ "\n",+ "Now we can instantiate our model object and generate chat completions:\n"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 19,+ "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",+ "metadata": {},+ "outputs": [],+ "source": [+ "from langchain_featherless_ai import ChatFeatherlessAi\n",+ "\n",+ "llm = ChatFeatherlessAi(\n",+ " model=\"featherless-ai/Qwerky-72B\",\n",+ " temperature=0.9,\n",+ " max_tokens=None,\n",+ " timeout=None,\n",+ ")"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "2b4f3e15",+ "metadata": {},+ "source": [+ "## Invocation"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 22,+ "id": "62e0dbc3",+ "metadata": {+ "tags": []+ },+ "outputs": [+ {+ "name": "stderr",+ "output_type": "stream",+ "text": [+ "c:\\Python311\\Lib\\site-packages\\pydantic\\main.py:463: UserWarning: Pydantic serializer warnings:\n",+ " PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [input_value=1747322408.706, input_type=float])\n",+ " return self.__pydantic_serializer__.to_python(\n"+ ]+ },+ {+ "data": {+ "text/plain": [+ "AIMessage(content=\"J'aime programmer.\", additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 5, 'prompt_tokens': 27, 'total_tokens': 32, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'featherless-ai/Qwerky-72B', 'system_fingerprint': '', 'id': 'G1sgui', 'service_tier': None, 'finish_reason': 'stop', 'logprobs': None}, id='run--6ecbe184-c94e-4d03-bf75-9bd85b04ba5b-0', usage_metadata={'input_tokens': 27, 'output_tokens': 5, 'total_tokens': 32, 'input_token_details': {}, 'output_token_details': {}})"+ ]+ },+ "execution_count": 22,+ "metadata": {},+ "output_type": "execute_result"+ }+ ],+ "source": [+ "messages = [\n",+ " (\n",+ " \"system\",\n",+ " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",+ " ),\n",+ " (\"human\", \"I love programming.\"),\n",+ "]\n",+ "ai_msg = llm.invoke(messages)\n",+ "ai_msg"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": 23,+ "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",+ "metadata": {},+ "outputs": [+ {+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "J'aime programmer.\n"+ ]+ }+ ],+ "source": [+ "print(ai_msg.content)"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",+ "metadata": {},+ "source": [+ "## Chaining\n",+ "\n",+ "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "fca9e713",+ "metadata": {},+ "outputs": [],+ "source": []+ },+ {+ "cell_type": "code",+ "execution_count": 24,+ "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",+ "metadata": {},+ "outputs": [+ {+ "name": "stderr",+ "output_type": "stream",+ "text": [+ "c:\\Python311\\Lib\\site-packages\\pydantic\\main.py:463: UserWarning: Pydantic serializer warnings:\n",+ " PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [input_value=1747322423.487, input_type=float])\n",+ " return self.__pydantic_serializer__.to_python(\n"+ ]+ },+ {+ "data": {+ "text/plain": [+ "AIMessage(content='Ich liebe Programmieren.', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 5, 'prompt_tokens': 22, 'total_tokens': 27, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'featherless-ai/Qwerky-72B', 'system_fingerprint': '', 'id': 'BoBqht', 'service_tier': None, 'finish_reason': 'stop', 'logprobs': None}, id='run--67464357-83d1-4591-9a62-303ed74b8148-0', usage_metadata={'input_tokens': 22, 'output_tokens': 5, 'total_tokens': 27, 'input_token_details': {}, 'output_token_details': {}})"+ ]+ },+ "execution_count": 24,+ "metadata": {},+ "output_type": "execute_result"+ }+ ],+ "source": [+ "from langchain_core.prompts import ChatPromptTemplate\n",+ "\n",+ "prompt = ChatPromptTemplate(\n",+ " [\n",+ " (\n",+ " \"system\",\n",+ " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",+ " ),\n",+ " (\"human\", \"{input}\"),\n",+ " ]\n",+ ")\n",+ "\n",+ "chain = prompt | llm\n",+ "chain.invoke(\n",+ " {\n",+ " \"input_language\": \"English\",\n",+ " \"output_language\": \"German\",\n",+ " \"input\": \"I love programming.\",\n",+ " }\n",+ ")"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",+ "metadata": {},+ "source": [+ "## API reference\n",+ "\n",+ "For detailed documentation of all ChatFeatherlessAi features and configurations head to the [API reference](https://python.langchain.com/api_reference/__package_name_short_snake__/chat_models/.chat_models.ChatFeatherlessAi.html)"+ ]+ }+ ],+ "metadata": {+ "kernelspec": {+ "display_name": "Python 3",+ "language": "python",+ "name": "python3"+ },+ "language_info": {+ "codemirror_mode": {+ "name": "ipython",+ "version": 3+ },+ "file_extension": ".py",+ "mimetype": "text/x-python",+ "name": "python",+ "nbconvert_exporter": "python",+ "pygments_lexer": "ipython3",+ "version": "3.11.3"+ }+ },+ "nbformat": 4,+ "nbformat_minor": 5+}diff --git a/docs/docs/integrations/providers/featherless-ai.mdx b/docs/docs/integrations/providers/featherless-ai.mdxnew file mode 100644index 0000000000..c10de11717--- /dev/null+++ b/docs/docs/integrations/providers/featherless-ai.mdx@@ -0,0 +1,12 @@+# Featherless AI++[Featherless AI](https://featherless.ai/) is a serverless AI inference platform that offers access to over 4300+ open-source models. Our goal is to make all AI models available for serverless inference. We provide inference via API to a continually expanding library of open-weight models.++# Installation and Setup+`pip install langchain-featherless-ai`+1. Sign up for an account at [Featherless](https://featherless.ai/register)+2. Subscribe to a plan and get your API key from [API Keys](https://featherless.ai/account/api-keys)+3. Set up your API key as an environment variable(`FEATHERLESSAI_API_KEY`)++# Model catalog+Visit our model catalog for an overview of all our models: https://featherless.ai/models\ No newline at end of filediff --git a/libs/packages.yml b/libs/packages.ymlindex ecf191635a..30cd92799f 100644--- a/libs/packages.yml+++ b/libs/packages.yml@@ -662,3 +662,6 @@ packages:- name: langchain-brightdatarepo: luminati-io/langchain-brightdatapath: .+- name: langchain-featherless-ai+ repo: featherlessai/langchain-featherless-ai+ path: .\ No newline at end of file