Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Add notebook using extractive qa to build structured data, closes #117

davidmezzetti6087b542d8d3
diff --git a/README.md b/README.md
index 27c5ac40..9053874d 100644
--- a/README.md
+++ b/README.md
@@ -107,6 +107,7 @@ NLP-backed data transformation pipelines and workflows.
|:----------|:-------------|------:|
| [Extractive QA with txtai](https://github.com/neuml/txtai/blob/master/examples/05_Extractive_QA_with_txtai.ipynb) | Introduction to extractive question-answering with txtai | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/05_Extractive_QA_with_txtai.ipynb) |
| [Extractive QA with Elasticsearch](https://github.com/neuml/txtai/blob/master/examples/06_Extractive_QA_with_Elasticsearch.ipynb) | Run extractive question-answering queries with Elasticsearch | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/06_Extractive_QA_with_Elasticsearch.ipynb) |
+| [Extractive QA to build structured data](https://github.com/neuml/txtai/blob/master/examples/20_Extractive_QA_to_build_structured_data.ipynb) | Build structured datasets using extractive question-answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/20_Extractive_QA_to_build_structured_data.ipynb) |
| [Apply labels with zero shot classification](https://github.com/neuml/txtai/blob/master/examples/07_Apply_labels_with_zero_shot_classification.ipynb) | Use zero shot learning for labeling, classification and topic modeling | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/07_Apply_labels_with_zero_shot_classification.ipynb) |
| [Building abstractive text summaries](https://github.com/neuml/txtai/blob/master/examples/09_Building_abstractive_text_summaries.ipynb) | Run abstractive text summarization | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/09_Building_abstractive_text_summaries.ipynb) |
| [Extract text from documents](https://github.com/neuml/txtai/blob/master/examples/10_Extract_text_from_documents.ipynb) | Extract text from PDF, Office, HTML and more | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/10_Extract_text_from_documents.ipynb) |
diff --git a/docs/examples.md b/docs/examples.md
index ccd6c778..a687049c 100644
--- a/docs/examples.md
+++ b/docs/examples.md
@@ -24,6 +24,7 @@ NLP-backed data transformation pipelines and workflows.
|:----------|:-------------|------:|
| [Extractive QA with txtai](https://github.com/neuml/txtai/blob/master/examples/05_Extractive_QA_with_txtai.ipynb) | Introduction to extractive question-answering with txtai | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/05_Extractive_QA_with_txtai.ipynb) |
| [Extractive QA with Elasticsearch](https://github.com/neuml/txtai/blob/master/examples/06_Extractive_QA_with_Elasticsearch.ipynb) | Run extractive question-answering queries with Elasticsearch | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/06_Extractive_QA_with_Elasticsearch.ipynb) |
+| [Extractive QA to build structured data](https://github.com/neuml/txtai/blob/master/examples/20_Extractive_QA_to_build_structured_data.ipynb) | Build structured datasets using extractive question-answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/20_Extractive_QA_to_build_structured_data.ipynb) |
| [Apply labels with zero shot classification](https://github.com/neuml/txtai/blob/master/examples/07_Apply_labels_with_zero_shot_classification.ipynb) | Use zero shot learning for labeling, classification and topic modeling | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/07_Apply_labels_with_zero_shot_classification.ipynb) |
| [Building abstractive text summaries](https://github.com/neuml/txtai/blob/master/examples/09_Building_abstractive_text_summaries.ipynb) | Run abstractive text summarization | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/09_Building_abstractive_text_summaries.ipynb) |
| [Extract text from documents](https://github.com/neuml/txtai/blob/master/examples/10_Extract_text_from_documents.ipynb) | Extract text from PDF, Office, HTML and more | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/10_Extract_text_from_documents.ipynb) |
diff --git a/examples/20_Extractive_QA_to_build_structured_data.ipynb b/examples/20_Extractive_QA_to_build_structured_data.ipynb
new file mode 100644
index 00000000..a733e6d1
--- /dev/null
+++ b/examples/20_Extractive_QA_to_build_structured_data.ipynb
@@ -0,0 +1,373 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+ "colab": {
+ "name": "20 - Extractive QA to build structured data",
+ "provenance": [],
+ "collapsed_sections": []
+ },
+ "kernelspec": {
+ "name": "python3",
+ "display_name": "Python 3"
+ }
+ },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "vwELCooy4ljr"
+ },
+ "source": [
+ "# Extractive QA to build structured data\n",
+ "\n",
+ "Traditional ETL/data parsing systems establish rules to extract information of interest. Regular expressions, string parsing and similar methods define fixed rules. This works in many cases but what if you are working with unstructured data containing numerous variations? The rules can be cumbersome and hard to maintain over time.\n",
+ "\n",
+ "This notebook uses machine learning and extractive question-answering (QA) to utilize the vast knowledge built into large language models. These models have been trained on extremely large datasets, learning the many variations of natural language. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ew7orE2O441o"
+ },
+ "source": [
+ "# Install dependencies\n",
+ "\n",
+ "Install `txtai` and all dependencies."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "LPQTb25tASIG"
+ },
+ "source": [
+ "%%capture\n",
+ "!pip install git+https://github.com/neuml/txtai"
+ ],
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_YnqorRKAbLu"
+ },
+ "source": [
+ "# Train a QA model with few-shot learning\n",
+ "\n",
+ "The code below trains a new QA model using a few examples. These examples gives the model hints on the type of questions that will be asked and the type of answers to look for. It doesn't take a lot of examples to do this as shown below."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "OUc9gqTyAYnm",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 75
+ },
+ "outputId": "b3a1d5ba-3176-4bfa-f294-633b33801f33"
+ },
+ "source": [
+ "import pandas as pd\n",
+ "from txtai.pipeline import HFTrainer, Questions, Labels\n",
+ "\n",
+ "# Training data for few-shot learning\n",
+ "data = [\n",
+ " {\"question\": \"What is the url?\",\n",
+ " \"context\": \"Faiss (https://github.com/facebookresearch/faiss) is a library for efficient similarity search.\",\n",
+ " \"answers\": \"https://github.com/facebookresearch/faiss\"},\n",
+ " {\"question\": \"What is the url\", \"context\": \"The last release was Wed Sept 25 2021\", \"answers\": None},\n",
+ " {\"question\": \"What is the date?\", \"context\": \"The last release was Wed Sept 25 2021\", \"answers\": \"Wed Sept 25 2021\"},\n",
+ " {\"question\": \"What is the date?\", \"context\": \"The order total comes to $44.33\", \"answers\": None},\n",
+ " {\"question\": \"What is the amount?\", \"context\": \"The order total comes to $44.33\", \"answers\": \"$44.33\"},\n",
+ " {\"question\": \"What is the amount?\", \"context\": \"The last release was Wed Sept 25 2021\", \"answers\": None},\n",
+ "]\n",
+ "\n",
+ "# Fine-tune QA model\n",
+ "trainer = HFTrainer()\n",
+ "model, tokenizer = trainer(\"distilbert-base-cased-distilled-squad\", data, task=\"question-answering\")"
+ ],
+ "execution_count": 10,
+ "outputs": [
+ {
+ "output_type": "display_data",
+ "data": {
+ "text/html": [
+ "\n",
+ " <div>\n",
+ " \n",
+ " <progress value='3' max='3' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+ " [3/3 00:03, Epoch 3/3]\n",
+ " </div>\n",
+ " <table border=\"1\" class=\"dataframe\">\n",
+ " <thead>\n",
+ " <tr style=\"text-align: left;\">\n",
+ " <th>Step</th>\n",
+ " <th>Training Loss</th>\n",
+ " </tr>\n",
+ " </thead>\n",
+ " <tbody>\n",
+ " </tbody>\n",
+ "</table><p>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML object>"
+ ]
+ },
+ "metadata": {}
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "1hzPnmrTaUjH"
+ },
+ "source": [
+ "# Parse data into a structured table\n",
+ "\n",
+ "The next section takes a series of rows of text and runs a set of questions against each row. The answers are then used to build a pandas DataFrame."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "id": "4X5z3UjnAGe7",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
+ },
+ "outputId": "1399c318-955e-49e8-c1d0-21456ce8a922"
+ },
+ "source": [
+ "# Input data\n",
+ "context = [\"Released on 6/03/2021\",\n",
+ " \"Release delayed until the 11th of August\",\n",
+ " \"Documentation can be found here: neuml.github.io/txtai\",\n",
+ " \"The stock price fell to three dollars\",\n",
+ " \"Great day: closing price for March 23rd is $33.11, for details - https://finance.google.com\"]\n",
+ "\n",
+ "# Define column queries\n",
+ "queries = [\"What is the url?\", \"What is the date?\", \"What is the amount?\"]\n",
+ "\n",
+ "# Extract fields\n",
+ "questions = Questions(path=(model, tokenizer), gpu=True)\n",
+ "results = [questions([question] * len(context), context) for question in queries]\n",
+ "results.append(context)\n",
+ "\n",
+ "# Load into DataFrame\n",
+ "pd.DataFrame(list(zip(*results)), columns=[\"URL\", \"Date\", \"Amount\", \"Text\"])"
+ ],
+ "execution_count": 11,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "<div>\n",
+ "<style scoped>\n",
+ " .dataframe tbody tr th:only-of-type {\n",
+ " vertical-align: middle;\n",
+ " }\n",
+ "\n",
+ " .dataframe tbody tr th {\n",
+ " vertical-align: top;\n",
+ " }\n",
+ "\n",
+ " .dataframe thead th {\n",
+ " text-align: right;\n",
+ " }\n",
+ "</style>\n",
+ "<table border=\"1\" class=\"dataframe\">\n",
+ " <thead>\n",
+ " <tr style=\"text-align: right;\">\n",
+ " <th></th>\n",
+ " <th>URL</th>\n",
+ " <th>Date</th>\n",
+ " <th>Amount</th>\n",
+ " <th>Text</th>\n",
+ " </tr>\n",
+ " </thead>\n",
+ " <tbody>\n",
+ " <tr>\n",
+ " <th>0</th>\n",
+ " <td>None</td>\n",
+ " <td>6/03/2021</td>\n",
+ " <td>None</td>\n",
+ " <td>Released on 6/03/2021</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>1</th>\n",
+ " <td>None</td>\n",
+ " <td>11th of August</td>\n",
+ " <td>None</td>\n",
+ " <td>Release delayed until the 11th of August</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>2</th>\n",
+ " <td>neuml.github.io/txtai</td>\n",
+ " <td>None</td>\n",
+ " <td>None</td>\n",
+ " <td>Documentation can be found here: neuml.github....</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>3</th>\n",
+ " <td>None</td>\n",
+ " <td>None</td>\n",
+ " <td>three dollars</td>\n",
+ " <td>The stock price fell to three dollars</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>4</th>\n",
+ " <td>https://finance.google.com</td>\n",
+ " <td>March 23rd</td>\n",
+ " <td>$33.11</td>\n",
+ " <td>Great day: closing price for March 23rd is $33...</td>\n",
+ " </tr>\n",
+ " </tbody>\n",
+ "</table>\n",
+ "</div>"
+ ],
+ "text/plain": [
+ " URL ... Text\n",
+ "0 None ... Released on 6/03/2021\n",
+ "1 None ... Release delayed until the 11th of August\n",
+ "2 neuml.github.io/txtai ... Documentation can be found here: neuml.github....\n",
+ "3 None ... The stock price fell to three dollars\n",
+ "4 https://finance.google.com ... Great day: closing price for March 23rd is $33...\n",
+ "\n",
+ "[5 rows x 4 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 11
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mY1Le-pve5yi"
+ },
+ "source": [
+ "# Add additional columns\n",
+ "\n",
+ "This method can be combined with other models to categorize, group or otherwise derive additional columns. The code below derives an additional sentiment column."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 206
+ },
+ "id": "0kyJUcrKe43a",
+ "outputId": "1b2dd3dd-d719-492f-e1b2-6cf737d4c9ea"
+ },
+ "source": [
+ "# Add sentiment\n",
+ "labels = Labels(path=\"distilbert-base-uncased-finetuned-sst-2-english\", dynamic=False)\n",
+ "labels = [\"POSITIVE\" if x[0][0] == 1 else \"NEGATIVE\" for x in labels(context)]\n",
+ "results.insert(len(results) - 1, labels)\n",
+ "\n",
+ "# Load into DataFrame\n",
+ "pd.DataFrame(list(zip(*results)), columns=[\"URL\", \"Date\", \"Amount\", \"Sentiment\", \"Text\"])"
+ ],
+ "execution_count": 12,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/html": [
+ "<div>\n",
+ "<style scoped>\n",
+ " .dataframe tbody tr th:only-of-type {\n",
+ " vertical-align: middle;\n",
+ " }\n",
+ "\n",
+ " .dataframe tbody tr th {\n",
+ " vertical-align: top;\n",
+ " }\n",
+ "\n",
+ " .dataframe thead th {\n",
+ " text-align: right;\n",
+ " }\n",
+ "</style>\n",
+ "<table border=\"1\" class=\"dataframe\">\n",
+ " <thead>\n",
+ " <tr style=\"text-align: right;\">\n",
+ " <th></th>\n",
+ " <th>URL</th>\n",
+ " <th>Date</th>\n",
+ " <th>Amount</th>\n",
+ " <th>Sentiment</th>\n",
+ " <th>Text</th>\n",
+ " </tr>\n",
+ " </thead>\n",
+ " <tbody>\n",
+ " <tr>\n",
+ " <th>0</th>\n",
+ " <td>None</td>\n",
+ " <td>6/03/2021</td>\n",
+ " <td>None</td>\n",
+ " <td>POSITIVE</td>\n",
+ " <td>Released on 6/03/2021</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>1</th>\n",
+ " <td>None</td>\n",
+ " <td>11th of August</td>\n",
+ " <td>None</td>\n",
+ " <td>NEGATIVE</td>\n",
+ " <td>Release delayed until the 11th of August</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>2</th>\n",
+ " <td>neuml.github.io/txtai</td>\n",
+ " <td>None</td>\n",
+ " <td>None</td>\n",
+ " <td>NEGATIVE</td>\n",
+ " <td>Documentation can be found here: neuml.github....</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>3</th>\n",
+ " <td>None</td>\n",
+ " <td>None</td>\n",
+ " <td>three dollars</td>\n",
+ " <td>NEGATIVE</td>\n",
+ " <td>The stock price fell to three dollars</td>\n",
+ " </tr>\n",
+ " <tr>\n",
+ " <th>4</th>\n",
+ " <td>https://finance.google.com</td>\n",
+ " <td>March 23rd</td>\n",
+ " <td>$33.11</td>\n",
+ " <td>POSITIVE</td>\n",
+ " <td>Great day: closing price for March 23rd is $33...</td>\n",
+ " </tr>\n",
+ " </tbody>\n",
+ "</table>\n",
+ "</div>"
+ ],
+ "text/plain": [
+ " URL ... Text\n",
+ "0 None ... Released on 6/03/2021\n",
+ "1 None ... Release delayed until the 11th of August\n",
+ "2 neuml.github.io/txtai ... Documentation can be found here: neuml.github....\n",
+ "3 None ... The stock price fell to three dollars\n",
+ "4 https://finance.google.com ... Great day: closing price for March 23rd is $33...\n",
+ "\n",
+ "[5 rows x 5 columns]"
+ ]
+ },
+ "metadata": {},
+ "execution_count": 12
+ }
+ ]
+ }
+ ]
+}
\ No newline at end of file