Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Update examples to use RAG pipeline, closes #766

davidmezzetti6eb5ceb79293
diff --git a/examples/01_Introducing_txtai.ipynb b/examples/01_Introducing_txtai.ipynb
index ee1e47f2..d8c6a246 100644
--- a/examples/01_Introducing_txtai.ipynb
+++ b/examples/01_Introducing_txtai.ipynb
@@ -733,7 +733,7 @@
"\n",
"txtai is an all-in-one embeddings database. It is the only vector database that also supports sparse indexes, graph networks and relational databases with inline SQL support. In addition to this, txtai has support for LLM orchestration.\n",
"\n",
- "The [extractor pipeline](https://neuml.github.io/txtai/pipeline/text/extractor/) is txtai's spin on retrieval augmented generation (RAG). This pipeline extracts knowledge from content by joining a prompt, context data store and generative model together.\n",
+ "The [RAG pipeline](https://neuml.github.io/txtai/pipeline/text/rag/) is txtai's spin on retrieval augmented generation (RAG). This pipeline extracts knowledge from content by joining a prompt, context data store and generative model together.\n",
"\n",
"The following example shows how a large language model (LLM) can use an embeddings database for context."
],
@@ -745,7 +745,7 @@
"cell_type": "code",
"source": [
"import torch\n",
- "from txtai.pipeline import Extractor\n",
+ "from txtai import RAG\n",
"\n",
"def prompt(question):\n",
" return [{\n",
@@ -763,9 +763,9 @@
"# Create an index for the list of text\n",
"embeddings.index(data)\n",
"\n",
- "# Create and run extractor instance\n",
- "extractor = Extractor(embeddings, \"google/flan-t5-large\", torch_dtype=torch.bfloat16, output=\"reference\")\n",
- "extractor(prompt(\"What country is having issues with climate change?\"))[0]"
+ "# Create and run RAG instance\n",
+ "rag = RAG(embeddings, \"google/flan-t5-large\", torch_dtype=torch.bfloat16, output=\"reference\")\n",
+ "rag(prompt(\"What country is having issues with climate change?\"))[0]"
],
"metadata": {
"colab": {
@@ -793,7 +793,7 @@
"source": [
"The logic above first builds an embeddings index. It then loads a LLM and uses the embeddings index to drive a LLM prompt.\n",
"\n",
- "The extractor pipeline can optionally return a reference to the id of the best matching record with the answer. That id can be used to resolve the full answer reference. Note that the embeddings above used an [uuid autosequence](https://neuml.github.io/txtai/embeddings/configuration/general/#autoid)."
+ "The RAG pipeline can optionally return a reference to the id of the best matching record with the answer. That id can be used to resolve the full answer reference. Note that the embeddings above used an [uuid autosequence](https://neuml.github.io/txtai/embeddings/configuration/general/#autoid)."
],
"metadata": {
"id": "lqsZreJQuSfO"
@@ -802,7 +802,7 @@
{
"cell_type": "code",
"source": [
- "uid = extractor(prompt(\"What country is having issues with climate change?\"))[0][\"reference\"]\n",
+ "uid = rag(prompt(\"What country is having issues with climate change?\"))[0][\"reference\"]\n",
"embeddings.search(f\"select id, text from txtai where id = '{uid}'\")"
],
"metadata": {
@@ -839,7 +839,7 @@
{
"cell_type": "code",
"source": [
- "from txtai.pipeline import LLM\n",
+ "from txtai import LLM\n",
"\n",
"llm = LLM(\"google/flan-t5-large\", torch_dtype=torch.bfloat16)\n",
"llm(\"Where is one place you'd go in Washington, DC?\")"
diff --git a/examples/42_Prompt_driven_search_with_LLMs.ipynb b/examples/42_Prompt_driven_search_with_LLMs.ipynb
index bc3237e6..4107f930 100644
--- a/examples/42_Prompt_driven_search_with_LLMs.ipynb
+++ b/examples/42_Prompt_driven_search_with_LLMs.ipynb
@@ -21,9 +21,9 @@
"source": [
"# Prompt-driven search with LLMs\n",
"\n",
- "This notebook revisits the Extractor pipeline, which has been covered in a number of previous notebooks. This pipeline is a combination of a similarity instance (embeddings or similarity pipeline) to build a question context and a model that answers questions.\n",
+ "This notebook revisits the RAG pipeline, which has been covered in a number of previous notebooks. This pipeline is a combination of a similarity instance (embeddings or similarity pipeline) to build a question context and a model that answers questions.\n",
"\n",
- "The Extractor pipeline recently underwent a number of major upgrades to support the following.\n",
+ "The RAG pipeline recently underwent a number of major upgrades to support the following.\n",
"\n",
"- Ability to run embeddings searches. Given that content is supported, text can be retrieved from the embeddings instance.\n",
"- In addition to extractive qa, support text generation models, sequence to sequence models and custom pipelines\n",
@@ -60,11 +60,11 @@
"id": "_YnqorRKAbLu"
},
"source": [
- "# Create Embeddings and Extractor instances\n",
+ "# Create Embeddings and RAG instances\n",
"\n",
"An Embeddings instance defines methods to represent text as vectors and build vector indexes for search.\n",
"\n",
- "The Extractor pipeline is a combination of a similarity instance (embeddings or similarity pipeline) to build a question context and a model that answers questions. The model can be a prompt-driven large language model (LLM), an extractive question-answering model or a custom pipeline.\n",
+ "The RAG pipeline is a combination of a similarity instance (embeddings or similarity pipeline) to build a question context and a model that answers questions. The model can be a prompt-driven large language model (LLM), an extractive question-answering model or a custom pipeline.\n",
"\n",
"Let's run a basic example.\n"
]
@@ -77,14 +77,13 @@
"source": [
"%%capture\n",
"\n",
- "from txtai.embeddings import Embeddings\n",
- "from txtai.pipeline import Extractor\n",
+ "from txtai import Embeddings, RAG\n",
"\n",
"# Create embeddings model with content support\n",
"embeddings = Embeddings({\"path\": \"sentence-transformers/all-MiniLM-L6-v2\", \"content\": True})\n",
"\n",
- "# Create extractor instance\n",
- "extractor = Extractor(embeddings, \"google/flan-t5-base\")"
+ "# Create RAG instance\n",
+ "rag = RAG(embeddings, \"google/flan-t5-base\")"
],
"execution_count": null,
"outputs": []
@@ -121,7 +120,7 @@
"\n",
"questions = [\"What team won the game?\", \"What was score?\"]\n",
"\n",
- "execute = lambda query: extractor([(question, query, prompt(question), False) for question in questions], data)\n",
+ "execute = lambda query: rag([(question, query, prompt(question), False) for question in questions], data)\n",
"\n",
"for query in [\"Red Sox - Blue Jays\", \"Phillies - Braves\", \"Dodgers - Giants\", \"Flyers - Lightning\"]:\n",
" print(\"----\", query, \"----\")\n",
@@ -191,8 +190,8 @@
"embeddings = Embeddings({\"path\": \"sentence-transformers/all-MiniLM-L6-v2\", \"content\": True})\n",
"embeddings.index((x, text, None) for x, text in enumerate(texts))\n",
"\n",
- "# Create extractor instance\n",
- "extractor = Extractor(embeddings, \"google/flan-t5-large\")"
+ "# Create RAG instance\n",
+ "rag = RAG(embeddings, \"google/flan-t5-large\")"
],
"metadata": {
"colab": {
@@ -201,16 +200,7 @@
"id": "yL716oEZ43t-",
"outputId": "23f4b0e7-a60a-4e89-fb57-06966e6612f8"
},
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stderr",
- "text": [
- "WARNING:datasets.builder:Found cached dataset ag_news (/root/.cache/huggingface/datasets/ag_news/default/0.0.0/bc2bcb40336ace1a0374767fc29bb0296cdaf8a6da7298436239c54d79180548)\n"
- ]
- }
- ]
+ "execution_count": null
},
{
"cell_type": "markdown",
@@ -234,7 +224,7 @@
" if not question:\n",
" question = query\n",
"\n",
- " return extractor([(\"answer\", query, prompt(question), False)])[0][1]\n",
+ " return rag([(\"answer\", query, prompt(question), False)])[0][1]\n",
"\n",
"question = \"Who won the 2004 presidential election?\"\n",
"answer = search(question)\n",
@@ -267,7 +257,7 @@
"source": [
"And there are the answers. Let's unpack how this works.\n",
"\n",
- "The first thing the Extractor pipeline does is run an embeddings search to find the most relevant text within the index. A context string is then built using those search results.\n",
+ "The first thing the RAG pipeline does is run an embeddings search to find the most relevant text within the index. A context string is then built using those search results.\n",
"\n",
"After that, a prompt is generated, run and the answer printed. Let's see what a full prompt looks like."
],
@@ -436,7 +426,7 @@
"source": [
"# External API Integration\n",
"\n",
- "In addition to support for Hugging Face models, the Extractor pipeline also supports custom question-answer models. This could be a call to the OpenAI API (GPT-3), Cohere API, Hugging Face API or using langchain to manage that. All that is needed is a Callable object or a function!\n",
+ "In addition to support for Hugging Face models, the RAG pipeline also supports custom question-answer models. This could be a call to the OpenAI API (GPT-3), Cohere API, Hugging Face API or using langchain to manage that. All that is needed is a Callable object or a function!\n",
"\n",
"Let's see an example that uses the Hugging Face API to answer questions. We'll use the original sports dataset to demonstrate."
],
@@ -480,12 +470,12 @@
"# Create embeddings model with content support\n",
"embeddings = Embeddings({\"path\": \"sentence-transformers/all-MiniLM-L6-v2\", \"content\": True})\n",
"\n",
- "# Create extractor instance, submit prompts to the Hugging Face inference API\n",
- "extractor = Extractor(embeddings, api)\n",
+ "# Create RAG instance, submit prompts to the Hugging Face inference API\n",
+ "rag = RAG(embeddings, api)\n",
"\n",
"questions = [\"What team won the game?\", \"What was score?\"]\n",
"\n",
- "execute = lambda query: extractor([(question, query, prompt(question), False) for question in questions], data)\n",
+ "execute = lambda query: rag([(question, query, prompt(question), False) for question in questions], data)\n",
"\n",
"for query in [\"Red Sox - Blue Jays\", \"Phillies - Braves\", \"Dodgers - Giants\", \"Flyers - Lightning\"]:\n",
" print(\"----\", query, \"----\")\n",
@@ -529,7 +519,7 @@
{
"cell_type": "markdown",
"source": [
- "Everything matches with first example above in [Create Embeddings and Extractor instances](#Create-Embeddings-and-Extractor-instances) except the prompts are run as an external API call.\n",
+ "Everything matches with first example above in [Create Embeddings and RAG instances](#Create-Embeddings-and-RAG-instances) except the prompts are run as an external API call.\n",
"\n",
"The Embeddings instance can also swap out the vectorization, database and vector store components with external API services. Check out the [txtai documentation](https://neuml.github.io/txtai/embeddings/configuration/) documentation for more information.\n",
"\n"
diff --git a/examples/44_Prompt_templates_and_task_chains.ipynb b/examples/44_Prompt_templates_and_task_chains.ipynb
index 144609fd..9e50d7fd 100644
--- a/examples/44_Prompt_templates_and_task_chains.ipynb
+++ b/examples/44_Prompt_templates_and_task_chains.ipynb
@@ -177,7 +177,7 @@
{
"cell_type": "code",
"source": [
- "from txtai.app import Application\n",
+ "from txtai import Application\n",
"\n",
"app = Application(\"workflow.yml\")\n",
"print(list(app.workflow(\"chain\", inputs)))"
@@ -340,20 +340,20 @@
" provider: huggingface-hub\n",
" container: neuml/txtai-intro\n",
"\n",
- "extractor:\n",
+ "rag:\n",
" path: google/flan-t5-large\n",
" output: reference\n",
"\n",
"workflow:\n",
" search:\n",
" tasks:\n",
- " - task: extractor\n",
+ " - task: rag\n",
" template: |\n",
" Answer the following question using only the context below. Give a detailed answer.\n",
" Say 'I don't have data on that' when the question can't be answered.\n",
" Question: {text}\n",
" Context: \n",
- " action: extractor\n",
+ " action: rag\n",
" - task: template\n",
" template: \"{answer}\\n\\nReference: {reference}\"\n",
" rules:\n",
diff --git a/examples/46_Whats_new_in_txtai_6_0.ipynb b/examples/46_Whats_new_in_txtai_6_0.ipynb
index 1c739c2b..e007715e 100644
--- a/examples/46_Whats_new_in_txtai_6_0.ipynb
+++ b/examples/46_Whats_new_in_txtai_6_0.ipynb
@@ -703,7 +703,7 @@
"source": [
"import torch\n",
"\n",
- "from txtai.pipeline import LLM\n",
+ "from txtai import LLM\n",
"\n",
"# Create model and set dtype to use 16-bit floats\n",
"llm = LLM(\"tiiuae/falcon-rw-1b\", torch_dtype=torch.bfloat16)"
diff --git a/examples/52_Build_RAG_pipelines_with_txtai.ipynb b/examples/52_Build_RAG_pipelines_with_txtai.ipynb
index 723a3f6b..77eb90a0 100644
--- a/examples/52_Build_RAG_pipelines_with_txtai.ipynb
+++ b/examples/52_Build_RAG_pipelines_with_txtai.ipynb
@@ -82,7 +82,7 @@
{
"cell_type": "code",
"source": [
- "from txtai.pipeline import LLM\n",
+ "from txtai import LLM\n",
"\n",
"# Create LLM\n",
"llm = LLM(\"TheBloke/Mistral-7B-OpenOrca-AWQ\")"
@@ -499,7 +499,7 @@
"source": [
"While the basic approach above works in this case, txtai has a more robust pipeline to handle citations and references.\n",
"\n",
- "The Extractor pipeline is defined below. An Extractor pipeline works in the same way as a LLM + Vector Search pipeline, except it has special logic for generating citations. This pipeline takes the answers and compares it to the context passed to the LLM to determine the most likely reference."
+ "The RAG pipeline is defined below. A RAG pipeline works in the same way as a LLM + Vector Search pipeline, except it has special logic for generating citations. This pipeline takes the answers and compares it to the context passed to the LLM to determine the most likely reference."
],
"metadata": {
"id": "MDcB7GCWY6TO"
@@ -508,9 +508,9 @@
{
"cell_type": "code",
"source": [
- "from txtai.pipeline import Extractor\n",
+ "from txtai import RAG\n",
"\n",
- "# Extractor prompt\n",
+ "# RAG prompt\n",
"def prompt(question):\n",
" return [{\n",
" \"query\": question,\n",
@@ -530,8 +530,8 @@
"<|im_start|>assistant\n",
"\"\"\")\n",
"\n",
- "# Create extractor instance\n",
- "extractor = Extractor(embeddings, llm, output=\"reference\")"
+ "# Create RAG instance\n",
+ "rag = RAG(embeddings, llm, output=\"reference\")"
],
"metadata": {
"id": "Lm6gg85_Y7ot"
@@ -542,7 +542,7 @@
{
"cell_type": "code",
"source": [
- "result = extractor(prompt(\"What version of Python is supported?\"), maxlength=4096, pad_token_id=32000)[0]\n",
+ "result = rag(prompt(\"What version of Python is supported?\"), maxlength=4096, pad_token_id=32000)[0]\n",
"print(\"ANSWER:\", result[\"answer\"])\n",
"print(\"CITATION:\", embeddings.search(\"select id, text from txtai where id = :id\", limit=1, parameters={\"id\": result[\"reference\"]}))"
],
@@ -568,7 +568,7 @@
{
"cell_type": "markdown",
"source": [
- "And as we can see, not only is the answer to the statement shown, the extractor pipeline also provides a citation. This step is crucial in any line of work where answers must be verified (which is most lines of work)."
+ "And as we can see, not only is the answer to the statement shown, the RAG pipeline also provides a citation. This step is crucial in any line of work where answers must be verified (which is most lines of work)."
],
"metadata": {
"id": "vHdE2Q59jNnF"
diff --git a/examples/53_Integrate_LLM_Frameworks.ipynb b/examples/53_Integrate_LLM_Frameworks.ipynb
index 2b217bad..7840d73c 100644
--- a/examples/53_Integrate_LLM_Frameworks.ipynb
+++ b/examples/53_Integrate_LLM_Frameworks.ipynb
@@ -72,8 +72,7 @@
{
"cell_type": "code",
"source": [
- "from txtai.embeddings import Embeddings\n",
- "from txtai.pipeline import Extractor, LLM\n",
+ "from txtai import Embeddings, RAG, LLM\n",
"\n",
"data = [\n",
" \"US tops 5 million confirmed virus cases\",\n",
@@ -109,9 +108,9 @@
"<|im_start|>assistant\n",
"\"\"\"\n",
"\n",
- "# Create and run extractor instance\n",
- "extractor = Extractor(embeddings, llm, output=\"reference\", separator=\"\\n\", template=template)\n",
- "result = extractor(\"Tell me about someone lucky\")\n",
+ "# Create and run RAG instance\n",
+ "rag = RAG(embeddings, llm, output=\"reference\", separator=\"\\n\", template=template)\n",
+ "result = rag(\"Tell me about someone lucky\")\n",
"\n",
"print(\"ANSWER:\", result[\"answer\"])\n",
"print(\"REFERENCE:\", embeddings.search(\"select id, text from txtai where id = :id\", parameters={\"id\": result[\"reference\"]}))\n"
@@ -283,9 +282,9 @@
"<|assistant|>\n",
"\"\"\"\n",
"\n",
- "# Create and run extractor instance\n",
- "extractor = Extractor(embeddings, llm, output=\"reference\", separator=\"\\n\", template=template)\n",
- "result = extractor(\"Tell me something about about wildlife\")\n",
+ "# Create and run RAG instance\n",
+ "rag = RAG(embeddings, llm, output=\"reference\", separator=\"\\n\", template=template)\n",
+ "result = rag(\"Tell me something about about wildlife\")\n",
"\n",
"print(\"ANSWER:\", result[\"answer\"])\n",
"print(\"REFERENCE:\", embeddings.search(\"select id, text from txtai where id = :id\", parameters={\"id\": result[\"reference\"]}))"
diff --git a/examples/55_Generate_knowledge_with_Semantic_Graphs_and_RAG.ipynb b/examples/55_Generate_knowledge_with_Semantic_Graphs_and_RAG.ipynb
index 83e1457b..19fe2f6a 100644
--- a/examples/55_Generate_knowledge_with_Semantic_Graphs_and_RAG.ipynb
+++ b/examples/55_Generate_knowledge_with_Semantic_Graphs_and_RAG.ipynb
@@ -295,7 +295,7 @@
},
"outputs": [],
"source": [
- "from txtai.pipeline import LLM\n",
+ "from txtai import LLM\n",
"llm = LLM(\"TheBloke/Mistral-7B-OpenOrca-AWQ\")"
]
},