Skip to content

The agent engineering platform.

docs: fireworks standard page (#23816)

Bagature787249af182
diff --git a/docs/docs/integrations/chat/fireworks.ipynb b/docs/docs/integrations/chat/fireworks.ipynb
index da519e8ba5..1e83aea7e3 100644
--- a/docs/docs/integrations/chat/fireworks.ipynb
+++ b/docs/docs/integrations/chat/fireworks.ipynb
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "raw",
- "id": "529aeba9",
+ "id": "afaf8039",
"metadata": {},
"source": [
"---\n",
@@ -11,197 +11,243 @@
]
},
{
- "attachments": {},
"cell_type": "markdown",
- "id": "642fd21c-600a-47a1-be96-6e1438b421a9",
+ "id": "e49f1e0d",
"metadata": {},
"source": [
"# ChatFireworks\n",
"\n",
- ">[Fireworks](https://app.fireworks.ai/) accelerates product development on generative AI by creating an innovative AI experiment and production platform. \n",
+ "This doc help you get started with Fireworks AI [chat models](/docs/concepts/#chat-models). For detailed documentation of all ChatFireworks features and configurations head to the [API reference](https://api.python.langchain.com/en/latest/chat_models/langchain_fireworks.chat_models.ChatFireworks.html).\n",
"\n",
- "This example goes over how to use LangChain to interact with `ChatFireworks` models."
+ "Fireworks AI is an AI inference platform to run and customize models. For a list of all models served by Fireworks see the [Fireworks docs](https://fireworks.ai/models).\n",
+ "\n",
+ "## Overview\n",
+ "### Integration details\n",
+ "\n",
+ "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/fireworks) | Package downloads | Package latest |\n",
+ "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
+ "| [ChatFireworks](https://api.python.langchain.com/en/latest/chat_models/langchain_fireworks.chat_models.ChatFireworks.html) | [langchain-fireworks](https://api.python.langchain.com/en/latest/fireworks_api_reference.html) | ❌ | beta | ✅ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-fireworks?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-fireworks?style=flat-square&label=%20) |\n",
+ "\n",
+ "### Model features\n",
+ "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
+ "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",
+ "| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | \n",
+ "\n",
+ "## Setup\n",
+ "\n",
+ "To access Fireworks models you'll need to create a Fireworks account, get an API key, and install the `langchain-fireworks` integration package.\n",
+ "\n",
+ "### Credentials\n",
+ "\n",
+ "Head to (ttps://fireworks.ai/login to sign up to Fireworks and generate an API key. Once you've done this set the FIREWORKS_API_KEY environment variable:"
]
},
{
- "cell_type": "raw",
- "id": "4a7c795e",
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "433e8d2b-9519-4b49-b2c4-7ab65b046c94",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import getpass\n",
+ "import os\n",
+ "\n",
+ "os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Enter your Fireworks API key: \")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "72ee0c4b-9764-423a-9dbf-95129e185210",
"metadata": {},
"source": [
- "%pip install langchain-fireworks"
+ "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:"
]
},
{
"cell_type": "code",
- "execution_count": 1,
- "id": "d00d850917865298",
- "metadata": {
- "collapsed": false,
- "jupyter": {
- "outputs_hidden": false
- }
- },
+ "execution_count": null,
+ "id": "a15d341e-3e26-4ca3-830b-5aab30ed66de",
+ "metadata": {},
"outputs": [],
"source": [
- "from langchain_core.messages import HumanMessage, SystemMessage\n",
- "from langchain_fireworks import ChatFireworks"
+ "# os.environ[\"LANGSMITH_API_KEY\"] = getpass.getpass(\"Enter your LangSmith API key: \")\n",
+ "# os.environ[\"LANGSMITH_TRACING\"] = \"true\""
]
},
{
"cell_type": "markdown",
- "id": "f28ebf8b-f14f-46c7-9962-8b8dc42e31be",
+ "id": "0730d6a1-c893-4840-9817-5e5251676d5d",
"metadata": {},
"source": [
- "# Setup\n",
+ "### Installation\n",
"\n",
- "1. Make sure the `langchain-fireworks` package is installed in your environment.\n",
- "2. Sign in to [Fireworks AI](http://fireworks.ai) for the an API Key to access our models, and make sure it is set as the `FIREWORKS_API_KEY` environment variable.\n",
- "3. Set up your model using a model id. If the model is not set, the default model is fireworks-llama-v2-7b-chat. See the full, most up-to-date model list on [app.fireworks.ai](https://app.fireworks.ai)."
+ "The LangChain Fireworks integration lives in the `langchain-fireworks` package:"
]
},
{
"cell_type": "code",
- "execution_count": 2,
- "id": "d096fb14-8acc-4047-9cd0-c842430c3a1d",
+ "execution_count": null,
+ "id": "652d6238-1f87-422a-b135-f5abbb8652fc",
"metadata": {},
"outputs": [],
"source": [
- "import getpass\n",
- "import os\n",
- "\n",
- "if \"FIREWORKS_API_KEY\" not in os.environ:\n",
- " os.environ[\"FIREWORKS_API_KEY\"] = getpass.getpass(\"Fireworks API Key:\")\n",
- "\n",
- "# Initialize a Fireworks chat model\n",
- "chat = ChatFireworks(model=\"accounts/fireworks/models/mixtral-8x7b-instruct\")"
+ "%pip install -qU langchain-fireworks"
]
},
{
"cell_type": "markdown",
- "id": "d8f13144-37cf-47a5-b5a0-e3cdf76d9a72",
+ "id": "a38cde65-254d-4219-a441-068766c0d4b5",
"metadata": {},
"source": [
- "# Calling the Model Directly\n",
+ "## Instantiation\n",
+ "\n",
+ "Now we can instantiate our model object and generate chat completions:\n",
"\n",
- "You can call the model directly with a system and human message to get answers."
+ "- TODO: Update model instantiation with relevant params."
]
},
{
"cell_type": "code",
- "execution_count": 3,
- "id": "72340871-ae2f-415f-b399-0777d32dc379",
+ "execution_count": 1,
+ "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
"metadata": {},
+ "outputs": [],
+ "source": [
+ "from langchain_fireworks import ChatFireworks\n",
+ "\n",
+ "llm = ChatFireworks(\n",
+ " model=\"accounts/fireworks/models/llama-v3-70b-instruct\",\n",
+ " temperature=0,\n",
+ " max_tokens=None,\n",
+ " timeout=None,\n",
+ " max_retries=2,\n",
+ " # other params...\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2b4f3e15",
+ "metadata": {},
+ "source": [
+ "## Invocation"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "62e0dbc3",
+ "metadata": {
+ "tags": []
+ },
"outputs": [
{
"data": {
"text/plain": [
- "AIMessage(content=\"Hello! I'm an AI language model, a helpful assistant designed to chat and assist you with any questions or information you might need. I'm here to make your experience as smooth and enjoyable as possible. How can I assist you today?\")"
+ "AIMessage(content=\"J'adore la programmation.\", response_metadata={'token_usage': {'prompt_tokens': 35, 'total_tokens': 44, 'completion_tokens': 9}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-df28e69a-ff30-457e-a743-06eb14d01cb0-0', usage_metadata={'input_tokens': 35, 'output_tokens': 9, 'total_tokens': 44})"
]
},
- "execution_count": 3,
+ "execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "# ChatFireworks Wrapper\n",
- "system_message = SystemMessage(content=\"You are to chat with the user.\")\n",
- "human_message = HumanMessage(content=\"Who are you?\")\n",
- "\n",
- "chat.invoke([system_message, human_message])"
+ "messages = [\n",
+ " (\n",
+ " \"system\",\n",
+ " \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
+ " ),\n",
+ " (\"human\", \"I love programming.\"),\n",
+ "]\n",
+ "ai_msg = llm.invoke(messages)\n",
+ "ai_msg"
]
},
{
"cell_type": "code",
- "execution_count": 5,
- "id": "68c6b1fa-2ff7-4a63-8d88-3cec302180b8",
+ "execution_count": 3,
+ "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
"metadata": {},
"outputs": [
{
- "data": {
- "text/plain": [
- "AIMessage(content=\"I'm an AI and do not have the ability to experience the weather firsthand. However,\")"
- ]
- },
- "execution_count": 5,
- "metadata": {},
- "output_type": "execute_result"
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "J'adore la programmation.\n"
+ ]
}
],
"source": [
- "# Setting additional parameters: temperature, max_tokens, top_p\n",
- "chat = ChatFireworks(\n",
- " model=\"accounts/fireworks/models/mixtral-8x7b-instruct\",\n",
- " temperature=1,\n",
- " max_tokens=20,\n",
- ")\n",
- "system_message = SystemMessage(content=\"You are to chat with the user.\")\n",
- "human_message = HumanMessage(content=\"How's the weather today?\")\n",
- "chat.invoke([system_message, human_message])"
+ "print(ai_msg.content)"
]
},
{
"cell_type": "markdown",
- "id": "8c44cb36",
+ "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",
"metadata": {},
"source": [
- "# Tool Calling\n",
+ "## Chaining\n",
"\n",
- "Fireworks offers the `FireFunction-v2` tool calling model. You can use it for structured output and function calling use cases:"
+ "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
]
},
{
"cell_type": "code",
- "execution_count": 11,
- "id": "ee2db682",
+ "execution_count": 4,
+ "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'function': {'arguments': '{\"name\": \"Erick\", \"age\": 27}',\n",
- " 'name': 'ExtractFields'},\n",
- " 'id': 'call_J0WYP2TLenaFw3UeVU0UnWqx',\n",
- " 'index': 0,\n",
- " 'type': 'function'}\n"
- ]
+ "data": {
+ "text/plain": [
+ "AIMessage(content='Ich liebe das Programmieren.', response_metadata={'token_usage': {'prompt_tokens': 30, 'total_tokens': 37, 'completion_tokens': 7}, 'model_name': 'accounts/fireworks/models/llama-v3-70b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-ff3f91ad-ed81-4acf-9f59-7490dc8d8f48-0', usage_metadata={'input_tokens': 30, 'output_tokens': 7, 'total_tokens': 37})"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
- "from pprint import pprint\n",
- "\n",
- "from langchain_core.pydantic_v1 import BaseModel\n",
- "\n",
- "\n",
- "class ExtractFields(BaseModel):\n",
- " name: str\n",
- " age: int\n",
+ "from langchain_core.prompts import ChatPromptTemplate\n",
"\n",
+ "prompt = ChatPromptTemplate.from_messages(\n",
+ " [\n",
+ " (\n",
+ " \"system\",\n",
+ " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
+ " ),\n",
+ " (\"human\", \"{input}\"),\n",
+ " ]\n",
+ ")\n",
"\n",
- "chat = ChatFireworks(\n",
- " model=\"accounts/fireworks/models/firefunction-v2\",\n",
- ").bind_tools([ExtractFields])\n",
- "\n",
- "result = chat.invoke(\"I am a 27 year old named Erick\")\n",
- "\n",
- "pprint(result.additional_kwargs[\"tool_calls\"][0])"
+ "chain = prompt | llm\n",
+ "chain.invoke(\n",
+ " {\n",
+ " \"input_language\": \"English\",\n",
+ " \"output_language\": \"German\",\n",
+ " \"input\": \"I love programming.\",\n",
+ " }\n",
+ ")"
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "id": "2321a4e6",
+ "cell_type": "markdown",
+ "id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
"metadata": {},
- "outputs": [],
- "source": []
+ "source": [
+ "## API reference\n",
+ "\n",
+ "For detailed documentation of all ChatFireworks features and configurations head to the API reference: https://api.python.langchain.com/en/latest/chat_models/langchain_fireworks.chat_models.ChatFireworks.html"
+ ]
}
],
"metadata": {
"kernelspec": {
- "display_name": "Python 3 (ipykernel)",
+ "display_name": "poetry-venv-2",
"language": "python",
- "name": "python3"
+ "name": "poetry-venv-2"
},
"language_info": {
"codemirror_mode": {
@@ -213,7 +259,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.11.4"
+ "version": "3.11.9"
}
},
"nbformat": 4,