DSPy: The framework for programming—not prompting—language models
Python
Complete run for LC-DSPy demo
Omar Khattabd863e853ef94
diff --git a/examples/tweets/compiling_langchain.ipynb b/examples/tweets/compiling_langchain.ipynbindex 17ac8b5ca..91b396108 100644--- a/examples/tweets/compiling_langchain.ipynb+++ b/examples/tweets/compiling_langchain.ipynb@@ -20,7 +20,7 @@},{"cell_type": "code",- "execution_count": null,+ "execution_count": 1,"metadata": {},"outputs": [],"source": [@@ -283,7 +283,7 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 63.999999999999986 / 150 (42.7): 100%|██████████| 150/150 [00:02<00:00, 65.97it/s]\n",+ "Average Metric: 63.999999999999986 / 150 (42.7): 100%|██████████| 150/150 [00:02<00:00, 65.78it/s]\n","/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dspy/evaluate/evaluate.py:126: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.\n"," df = df.applymap(truncate_cell)\n"]@@ -299,82 +299,82 @@"data": {"text/html": ["<style type=\"text/css\">\n",- "#T_042b5 th {\n",+ "#T_ed282 th {\n"," text-align: left;\n","}\n",- "#T_042b5 td {\n",+ "#T_ed282 td {\n"," text-align: left;\n","}\n",- "#T_042b5_row0_col0, #T_042b5_row0_col1, #T_042b5_row0_col2, #T_042b5_row0_col3, #T_042b5_row0_col4, #T_042b5_row0_col5, #T_042b5_row1_col0, #T_042b5_row1_col1, #T_042b5_row1_col2, #T_042b5_row1_col3, #T_042b5_row1_col4, #T_042b5_row1_col5, #T_042b5_row2_col0, #T_042b5_row2_col1, #T_042b5_row2_col2, #T_042b5_row2_col3, #T_042b5_row2_col4, #T_042b5_row2_col5, #T_042b5_row3_col0, #T_042b5_row3_col1, #T_042b5_row3_col2, #T_042b5_row3_col3, #T_042b5_row3_col4, #T_042b5_row3_col5, #T_042b5_row4_col0, #T_042b5_row4_col1, #T_042b5_row4_col2, #T_042b5_row4_col3, #T_042b5_row4_col4, #T_042b5_row4_col5 {\n",+ "#T_ed282_row0_col0, #T_ed282_row0_col1, #T_ed282_row0_col2, #T_ed282_row0_col3, #T_ed282_row0_col4, #T_ed282_row0_col5, #T_ed282_row1_col0, #T_ed282_row1_col1, #T_ed282_row1_col2, #T_ed282_row1_col3, #T_ed282_row1_col4, #T_ed282_row1_col5, #T_ed282_row2_col0, #T_ed282_row2_col1, #T_ed282_row2_col2, #T_ed282_row2_col3, #T_ed282_row2_col4, #T_ed282_row2_col5, #T_ed282_row3_col0, #T_ed282_row3_col1, #T_ed282_row3_col2, #T_ed282_row3_col3, #T_ed282_row3_col4, #T_ed282_row3_col5, #T_ed282_row4_col0, #T_ed282_row4_col1, #T_ed282_row4_col2, #T_ed282_row4_col3, #T_ed282_row4_col4, #T_ed282_row4_col5 {\n"," text-align: left;\n"," white-space: pre-wrap;\n"," word-wrap: break-word;\n"," max-width: 400px;\n","}\n","</style>\n",- "<table id=\"T_042b5\">\n",+ "<table id=\"T_ed282\">\n"," <thead>\n"," <tr>\n"," <th class=\"blank level0\" > </th>\n",- " <th id=\"T_042b5_level0_col0\" class=\"col_heading level0 col0\" >question</th>\n",- " <th id=\"T_042b5_level0_col1\" class=\"col_heading level0 col1\" >answer</th>\n",- " <th id=\"T_042b5_level0_col2\" class=\"col_heading level0 col2\" >gold_titles</th>\n",- " <th id=\"T_042b5_level0_col3\" class=\"col_heading level0 col3\" >output</th>\n",- " <th id=\"T_042b5_level0_col4\" class=\"col_heading level0 col4\" >tweet_response</th>\n",- " <th id=\"T_042b5_level0_col5\" class=\"col_heading level0 col5\" >metric</th>\n",+ " <th id=\"T_ed282_level0_col0\" class=\"col_heading level0 col0\" >question</th>\n",+ " <th id=\"T_ed282_level0_col1\" class=\"col_heading level0 col1\" >answer</th>\n",+ " <th id=\"T_ed282_level0_col2\" class=\"col_heading level0 col2\" >gold_titles</th>\n",+ " <th id=\"T_ed282_level0_col3\" class=\"col_heading level0 col3\" >output</th>\n",+ " <th id=\"T_ed282_level0_col4\" class=\"col_heading level0 col4\" >tweet_response</th>\n",+ " <th id=\"T_ed282_level0_col5\" class=\"col_heading level0 col5\" >metric</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n",- " <th id=\"T_042b5_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",- " <td id=\"T_042b5_row0_col0\" class=\"data row0 col0\" >Who was a producer who produced albums for both rock bands Juke Karten and Thirty Seconds to Mars?</td>\n",- " <td id=\"T_042b5_row0_col1\" class=\"data row0 col1\" >Brian Virtue</td>\n",- " <td id=\"T_042b5_row0_col2\" class=\"data row0 col2\" >{'Levolution (album)', 'Thirty Seconds to Mars'}</td>\n",- " <td id=\"T_042b5_row0_col3\" class=\"data row0 col3\" >Brian Virtue, who has worked with bands like Jane's Addiction and Velvet Revolver, produced albums for both Juke Kartel and Thirty Seconds to Mars, showcasing...</td>\n",- " <td id=\"T_042b5_row0_col4\" class=\"data row0 col4\" >Brian Virtue, who has worked with bands like Jane's Addiction and Velvet Revolver, produced albums for both Juke Kartel and Thirty Seconds to Mars, showcasing...</td>\n",- " <td id=\"T_042b5_row0_col5\" class=\"data row0 col5\" >1.0</td>\n",+ " <th id=\"T_ed282_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",+ " <td id=\"T_ed282_row0_col0\" class=\"data row0 col0\" >Who was a producer who produced albums for both rock bands Juke Karten and Thirty Seconds to Mars?</td>\n",+ " <td id=\"T_ed282_row0_col1\" class=\"data row0 col1\" >Brian Virtue</td>\n",+ " <td id=\"T_ed282_row0_col2\" class=\"data row0 col2\" >{'Levolution (album)', 'Thirty Seconds to Mars'}</td>\n",+ " <td id=\"T_ed282_row0_col3\" class=\"data row0 col3\" >Brian Virtue, who has worked with bands like Jane's Addiction and Velvet Revolver, produced albums for both Juke Kartel and Thirty Seconds to Mars, showcasing...</td>\n",+ " <td id=\"T_ed282_row0_col4\" class=\"data row0 col4\" >Brian Virtue, who has worked with bands like Jane's Addiction and Velvet Revolver, produced albums for both Juke Kartel and Thirty Seconds to Mars, showcasing...</td>\n",+ " <td id=\"T_ed282_row0_col5\" class=\"data row0 col5\" >1.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_042b5_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",- " <td id=\"T_042b5_row1_col0\" class=\"data row1 col0\" >Are both the University of Chicago and Syracuse University public universities? </td>\n",- " <td id=\"T_042b5_row1_col1\" class=\"data row1 col1\" >no</td>\n",- " <td id=\"T_042b5_row1_col2\" class=\"data row1 col2\" >{'University of Chicago', 'Syracuse University'}</td>\n",- " <td id=\"T_042b5_row1_col3\" class=\"data row1 col3\" > No, only Syracuse University is a public university. The University of Chicago is a private research university. #Syracuse #University #Chicago #Public #Private</td>\n",- " <td id=\"T_042b5_row1_col4\" class=\"data row1 col4\" > No, only Syracuse University is a public university. The University of Chicago is a private research university. #Syracuse #University #Chicago #Public #Private</td>\n",- " <td id=\"T_042b5_row1_col5\" class=\"data row1 col5\" >0.3333333333333333</td>\n",+ " <th id=\"T_ed282_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",+ " <td id=\"T_ed282_row1_col0\" class=\"data row1 col0\" >Are both the University of Chicago and Syracuse University public universities? </td>\n",+ " <td id=\"T_ed282_row1_col1\" class=\"data row1 col1\" >no</td>\n",+ " <td id=\"T_ed282_row1_col2\" class=\"data row1 col2\" >{'Syracuse University', 'University of Chicago'}</td>\n",+ " <td id=\"T_ed282_row1_col3\" class=\"data row1 col3\" > No, only Syracuse University is a public university. The University of Chicago is a private research university. #Syracuse #University #Chicago #Public #Private</td>\n",+ " <td id=\"T_ed282_row1_col4\" class=\"data row1 col4\" > No, only Syracuse University is a public university. The University of Chicago is a private research university. #Syracuse #University #Chicago #Public #Private</td>\n",+ " <td id=\"T_ed282_row1_col5\" class=\"data row1 col5\" >0.3333333333333333</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_042b5_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",- " <td id=\"T_042b5_row2_col0\" class=\"data row2 col0\" >In what region was Eddy Mazzoleni born?</td>\n",- " <td id=\"T_042b5_row2_col1\" class=\"data row2 col1\" >Lombardy, northern Italy</td>\n",- " <td id=\"T_042b5_row2_col2\" class=\"data row2 col2\" >{'Bergamo', 'Eddy Mazzoleni'}</td>\n",- " <td id=\"T_042b5_row2_col3\" class=\"data row2 col3\" > Eddy Mazzoleni, Italian professional cyclist, was born in Bergamo, Italy on July 29, 1973. #cyclist #Italy #Bergamo</td>\n",- " <td id=\"T_042b5_row2_col4\" class=\"data row2 col4\" > Eddy Mazzoleni, Italian professional cyclist, was born in Bergamo, Italy on July 29, 1973. #cyclist #Italy #Bergamo</td>\n",- " <td id=\"T_042b5_row2_col5\" class=\"data row2 col5\" >0.0</td>\n",+ " <th id=\"T_ed282_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",+ " <td id=\"T_ed282_row2_col0\" class=\"data row2 col0\" >In what region was Eddy Mazzoleni born?</td>\n",+ " <td id=\"T_ed282_row2_col1\" class=\"data row2 col1\" >Lombardy, northern Italy</td>\n",+ " <td id=\"T_ed282_row2_col2\" class=\"data row2 col2\" >{'Bergamo', 'Eddy Mazzoleni'}</td>\n",+ " <td id=\"T_ed282_row2_col3\" class=\"data row2 col3\" > Eddy Mazzoleni, Italian professional cyclist, was born in Bergamo, Italy on July 29, 1973. #cyclist #Italy #Bergamo</td>\n",+ " <td id=\"T_ed282_row2_col4\" class=\"data row2 col4\" > Eddy Mazzoleni, Italian professional cyclist, was born in Bergamo, Italy on July 29, 1973. #cyclist #Italy #Bergamo</td>\n",+ " <td id=\"T_ed282_row2_col5\" class=\"data row2 col5\" >0.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_042b5_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",- " <td id=\"T_042b5_row3_col0\" class=\"data row3 col0\" >Who edited the 1990 American romantic comedy film directed by Garry Marshall?</td>\n",- " <td id=\"T_042b5_row3_col1\" class=\"data row3 col1\" >Raja Raymond Gosnell</td>\n",- " <td id=\"T_042b5_row3_col2\" class=\"data row3 col2\" >{'Raja Gosnell', 'Pretty Woman'}</td>\n",- " <td id=\"T_042b5_row3_col3\" class=\"data row3 col3\" > J. F. Lawton edited the 1990 American romantic comedy film directed by Garry Marshall. #PrettyWoman #GarryMarshall #JFLawton</td>\n",- " <td id=\"T_042b5_row3_col4\" class=\"data row3 col4\" > J. F. Lawton edited the 1990 American romantic comedy film directed by Garry Marshall. #PrettyWoman #GarryMarshall #JFLawton</td>\n",- " <td id=\"T_042b5_row3_col5\" class=\"data row3 col5\" >0.0</td>\n",+ " <th id=\"T_ed282_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",+ " <td id=\"T_ed282_row3_col0\" class=\"data row3 col0\" >Who edited the 1990 American romantic comedy film directed by Garry Marshall?</td>\n",+ " <td id=\"T_ed282_row3_col1\" class=\"data row3 col1\" >Raja Raymond Gosnell</td>\n",+ " <td id=\"T_ed282_row3_col2\" class=\"data row3 col2\" >{'Raja Gosnell', 'Pretty Woman'}</td>\n",+ " <td id=\"T_ed282_row3_col3\" class=\"data row3 col3\" > J. F. Lawton edited the 1990 American romantic comedy film directed by Garry Marshall. #PrettyWoman #GarryMarshall #JFLawton</td>\n",+ " <td id=\"T_ed282_row3_col4\" class=\"data row3 col4\" > J. F. Lawton edited the 1990 American romantic comedy film directed by Garry Marshall. #PrettyWoman #GarryMarshall #JFLawton</td>\n",+ " <td id=\"T_ed282_row3_col5\" class=\"data row3 col5\" >0.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_042b5_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",- " <td id=\"T_042b5_row4_col0\" class=\"data row4 col0\" >Burrs Country Park railway station is what stop on the railway line that runs between Heywood and Rawtenstall</td>\n",- " <td id=\"T_042b5_row4_col1\" class=\"data row4 col1\" >seventh</td>\n",- " <td id=\"T_042b5_row4_col2\" class=\"data row4 col2\" >{'East Lancashire Railway', 'Burrs Country Park railway station'}</td>\n",- " <td id=\"T_042b5_row4_col3\" class=\"data row4 col3\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line that runs between Heywood and Rawtenstall.</td>\n",- " <td id=\"T_042b5_row4_col4\" class=\"data row4 col4\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line that runs between Heywood and Rawtenstall.</td>\n",- " <td id=\"T_042b5_row4_col5\" class=\"data row4 col5\" >1.0</td>\n",+ " <th id=\"T_ed282_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",+ " <td id=\"T_ed282_row4_col0\" class=\"data row4 col0\" >Burrs Country Park railway station is what stop on the railway line that runs between Heywood and Rawtenstall</td>\n",+ " <td id=\"T_ed282_row4_col1\" class=\"data row4 col1\" >seventh</td>\n",+ " <td id=\"T_ed282_row4_col2\" class=\"data row4 col2\" >{'East Lancashire Railway', 'Burrs Country Park railway station'}</td>\n",+ " <td id=\"T_ed282_row4_col3\" class=\"data row4 col3\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line that runs between Heywood and Rawtenstall.</td>\n",+ " <td id=\"T_ed282_row4_col4\" class=\"data row4 col4\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line that runs between Heywood and Rawtenstall.</td>\n",+ " <td id=\"T_ed282_row4_col5\" class=\"data row4 col5\" >1.0</td>\n"," </tr>\n"," </tbody>\n","</table>\n"],"text/plain": [- "<pandas.io.formats.style.Styler at 0x7f08b9f75490>"+ "<pandas.io.formats.style.Styler at 0x7f8c58f5f6d0>"]},"metadata": {},@@ -460,7 +460,7 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 22.333333333333336 / 50 (44.7): 100%|██████████| 50/50 [00:00<00:00, 56.06it/s]\n",+ "Average Metric: 22.333333333333336 / 50 (44.7): 100%|██████████| 50/50 [00:00<00:00, 55.76it/s]\n","/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dspy/evaluate/evaluate.py:126: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.\n"," df = df.applymap(truncate_cell)\n"]@@ -480,7 +480,7 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 22.333333333333336 / 50 (44.7): 100%|██████████| 50/50 [00:00<00:00, 126.78it/s]\n",+ "Average Metric: 22.333333333333332 / 50 (44.7): 100%|██████████| 50/50 [00:00<00:00, 224.17it/s]\n","/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dspy/evaluate/evaluate.py:126: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.\n"," df = df.applymap(truncate_cell)\n"]@@ -489,7 +489,7 @@"name": "stdout","output_type": "stream","text": [- "Average Metric: 22.333333333333336 / 50 (44.7%)\n",+ "Average Metric: 22.333333333333332 / 50 (44.7%)\n","Score: 44.67 for set: [16]\n","Scores so far: [44.67, 44.67]\n","Best score: 44.67\n"@@ -499,7 +499,7 @@"name": "stderr","output_type": "stream","text": [- " 2%|▎ | 5/200 [00:00<00:07, 26.22it/s]\n"+ " 2%|▎ | 5/200 [00:00<00:07, 27.18it/s]\n"]},{@@ -513,7 +513,7 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 27.000000000000004 / 50 (54.0): 100%|██████████| 50/50 [00:00<00:00, 70.41it/s]\n",+ "Average Metric: 27.000000000000004 / 50 (54.0): 100%|██████████| 50/50 [00:00<00:00, 71.93it/s]\n","/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dspy/evaluate/evaluate.py:126: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.\n"," df = df.applymap(truncate_cell)\n"]@@ -539,7 +539,7 @@"name": "stderr","output_type": "stream","text": [- " 4%|▍ | 9/200 [00:00<00:06, 28.02it/s]\n"+ " 4%|▍ | 9/200 [00:00<00:06, 27.43it/s]\n"]},{@@ -553,7 +553,7 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 25.000000000000004 / 50 (50.0): 100%|██████████| 50/50 [00:00<00:00, 76.56it/s]\n",+ "Average Metric: 25.000000000000004 / 50 (50.0): 100%|██████████| 50/50 [00:00<00:00, 74.02it/s]\n","/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dspy/evaluate/evaluate.py:126: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.\n"," df = df.applymap(truncate_cell)\n"]@@ -578,7 +578,7 @@"name": "stderr","output_type": "stream","text": [- " 0%| | 1/200 [00:00<00:06, 28.50it/s]\n"+ " 0%| | 1/200 [00:00<00:07, 27.41it/s]\n"]},{@@ -592,7 +592,7 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 25.666666666666664 / 50 (51.3): 100%|██████████| 50/50 [00:00<00:00, 76.77it/s]\n",+ "Average Metric: 25.666666666666668 / 50 (51.3): 100%|██████████| 50/50 [00:00<00:00, 78.28it/s]\n","/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dspy/evaluate/evaluate.py:126: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.\n"," df = df.applymap(truncate_cell)\n"]@@ -601,7 +601,7 @@"name": "stdout","output_type": "stream","text": [- "Average Metric: 25.666666666666664 / 50 (51.3%)\n",+ "Average Metric: 25.666666666666668 / 50 (51.3%)\n","Score: 51.33 for set: [16]\n","Scores so far: [44.67, 44.67, 54.0, 50.0, 51.33]\n","Best score: 54.0\n",@@ -617,7 +617,7 @@"name": "stderr","output_type": "stream","text": [- " 1%| | 2/200 [00:00<00:07, 27.13it/s]\n"+ " 1%| | 2/200 [00:00<00:07, 26.31it/s]\n"]},{@@ -631,14 +631,14 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 26.0 / 50 (52.0): 100%|██████████| 50/50 [00:00<00:00, 73.05it/s] "+ "Average Metric: 26.000000000000004 / 50 (52.0): 100%|██████████| 50/50 [00:00<00:00, 74.75it/s]"]},{"name": "stdout","output_type": "stream","text": [- "Average Metric: 26.0 / 50 (52.0%)\n",+ "Average Metric: 26.000000000000004 / 50 (52.0%)\n","Score: 52.0 for set: [16]\n","Scores so far: [44.67, 44.67, 54.0, 50.0, 51.33, 52.0]\n","Best score: 54.0\n",@@ -719,14 +719,14 @@"name": "stderr","output_type": "stream","text": [- "Average Metric: 78.66666666666666 / 150 (52.4): 100%|██████████| 150/150 [00:02<00:00, 74.17it/s] "+ "Average Metric: 78.66666666666667 / 150 (52.4): 100%|██████████| 150/150 [00:02<00:00, 71.07it/s] "]},{"name": "stdout","output_type": "stream","text": [- "Average Metric: 78.66666666666666 / 150 (52.4%)\n"+ "Average Metric: 78.66666666666667 / 150 (52.4%)\n"]},{@@ -742,82 +742,82 @@"data": {"text/html": ["<style type=\"text/css\">\n",- "#T_c36cc th {\n",+ "#T_c3dbe th {\n"," text-align: left;\n","}\n",- "#T_c36cc td {\n",+ "#T_c3dbe td {\n"," text-align: left;\n","}\n",- "#T_c36cc_row0_col0, #T_c36cc_row0_col1, #T_c36cc_row0_col2, #T_c36cc_row0_col3, #T_c36cc_row0_col4, #T_c36cc_row0_col5, #T_c36cc_row1_col0, #T_c36cc_row1_col1, #T_c36cc_row1_col2, #T_c36cc_row1_col3, #T_c36cc_row1_col4, #T_c36cc_row1_col5, #T_c36cc_row2_col0, #T_c36cc_row2_col1, #T_c36cc_row2_col2, #T_c36cc_row2_col3, #T_c36cc_row2_col4, #T_c36cc_row2_col5, #T_c36cc_row3_col0, #T_c36cc_row3_col1, #T_c36cc_row3_col2, #T_c36cc_row3_col3, #T_c36cc_row3_col4, #T_c36cc_row3_col5, #T_c36cc_row4_col0, #T_c36cc_row4_col1, #T_c36cc_row4_col2, #T_c36cc_row4_col3, #T_c36cc_row4_col4, #T_c36cc_row4_col5 {\n",+ "#T_c3dbe_row0_col0, #T_c3dbe_row0_col1, #T_c3dbe_row0_col2, #T_c3dbe_row0_col3, #T_c3dbe_row0_col4, #T_c3dbe_row0_col5, #T_c3dbe_row1_col0, #T_c3dbe_row1_col1, #T_c3dbe_row1_col2, #T_c3dbe_row1_col3, #T_c3dbe_row1_col4, #T_c3dbe_row1_col5, #T_c3dbe_row2_col0, #T_c3dbe_row2_col1, #T_c3dbe_row2_col2, #T_c3dbe_row2_col3, #T_c3dbe_row2_col4, #T_c3dbe_row2_col5, #T_c3dbe_row3_col0, #T_c3dbe_row3_col1, #T_c3dbe_row3_col2, #T_c3dbe_row3_col3, #T_c3dbe_row3_col4, #T_c3dbe_row3_col5, #T_c3dbe_row4_col0, #T_c3dbe_row4_col1, #T_c3dbe_row4_col2, #T_c3dbe_row4_col3, #T_c3dbe_row4_col4, #T_c3dbe_row4_col5 {\n"," text-align: left;\n"," white-space: pre-wrap;\n"," word-wrap: break-word;\n"," max-width: 400px;\n","}\n","</style>\n",- "<table id=\"T_c36cc\">\n",+ "<table id=\"T_c3dbe\">\n"," <thead>\n"," <tr>\n"," <th class=\"blank level0\" > </th>\n",- " <th id=\"T_c36cc_level0_col0\" class=\"col_heading level0 col0\" >question</th>\n",- " <th id=\"T_c36cc_level0_col1\" class=\"col_heading level0 col1\" >answer</th>\n",- " <th id=\"T_c36cc_level0_col2\" class=\"col_heading level0 col2\" >gold_titles</th>\n",- " <th id=\"T_c36cc_level0_col3\" class=\"col_heading level0 col3\" >output</th>\n",- " <th id=\"T_c36cc_level0_col4\" class=\"col_heading level0 col4\" >tweet_response</th>\n",- " <th id=\"T_c36cc_level0_col5\" class=\"col_heading level0 col5\" >metric</th>\n",+ " <th id=\"T_c3dbe_level0_col0\" class=\"col_heading level0 col0\" >question</th>\n",+ " <th id=\"T_c3dbe_level0_col1\" class=\"col_heading level0 col1\" >answer</th>\n",+ " <th id=\"T_c3dbe_level0_col2\" class=\"col_heading level0 col2\" >gold_titles</th>\n",+ " <th id=\"T_c3dbe_level0_col3\" class=\"col_heading level0 col3\" >output</th>\n",+ " <th id=\"T_c3dbe_level0_col4\" class=\"col_heading level0 col4\" >tweet_response</th>\n",+ " <th id=\"T_c3dbe_level0_col5\" class=\"col_heading level0 col5\" >metric</th>\n"," </tr>\n"," </thead>\n"," <tbody>\n"," <tr>\n",- " <th id=\"T_c36cc_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",- " <td id=\"T_c36cc_row0_col0\" class=\"data row0 col0\" >Who was a producer who produced albums for both rock bands Juke Karten and Thirty Seconds to Mars?</td>\n",- " <td id=\"T_c36cc_row0_col1\" class=\"data row0 col1\" >Brian Virtue</td>\n",- " <td id=\"T_c36cc_row0_col2\" class=\"data row0 col2\" >{'Levolution (album)', 'Thirty Seconds to Mars'}</td>\n",- " <td id=\"T_c36cc_row0_col3\" class=\"data row0 col3\" >Brian Virtue is a producer who has worked with both Juke Kartel and Thirty Seconds to Mars, helping to create their unique sounds. #BrianVirtue #producer...</td>\n",- " <td id=\"T_c36cc_row0_col4\" class=\"data row0 col4\" >Brian Virtue is a producer who has worked with both Juke Kartel and Thirty Seconds to Mars, helping to create their unique sounds. #BrianVirtue #producer...</td>\n",- " <td id=\"T_c36cc_row0_col5\" class=\"data row0 col5\" >1.0</td>\n",+ " <th id=\"T_c3dbe_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",+ " <td id=\"T_c3dbe_row0_col0\" class=\"data row0 col0\" >Who was a producer who produced albums for both rock bands Juke Karten and Thirty Seconds to Mars?</td>\n",+ " <td id=\"T_c3dbe_row0_col1\" class=\"data row0 col1\" >Brian Virtue</td>\n",+ " <td id=\"T_c3dbe_row0_col2\" class=\"data row0 col2\" >{'Levolution (album)', 'Thirty Seconds to Mars'}</td>\n",+ " <td id=\"T_c3dbe_row0_col3\" class=\"data row0 col3\" >Brian Virtue is a producer who has worked with both Juke Kartel and Thirty Seconds to Mars, helping to create their unique sounds. #BrianVirtue #producer...</td>\n",+ " <td id=\"T_c3dbe_row0_col4\" class=\"data row0 col4\" >Brian Virtue is a producer who has worked with both Juke Kartel and Thirty Seconds to Mars, helping to create their unique sounds. #BrianVirtue #producer...</td>\n",+ " <td id=\"T_c3dbe_row0_col5\" class=\"data row0 col5\" >1.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_c36cc_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",- " <td id=\"T_c36cc_row1_col0\" class=\"data row1 col0\" >Are both the University of Chicago and Syracuse University public universities? </td>\n",- " <td id=\"T_c36cc_row1_col1\" class=\"data row1 col1\" >no</td>\n",- " <td id=\"T_c36cc_row1_col2\" class=\"data row1 col2\" >{'University of Chicago', 'Syracuse University'}</td>\n",- " <td id=\"T_c36cc_row1_col3\" class=\"data row1 col3\" > Yes, both Northeastern Illinois University and Syracuse University are public universities. #publicuniversity #Chicago #Syracuse</td>\n",- " <td id=\"T_c36cc_row1_col4\" class=\"data row1 col4\" > Yes, both Northeastern Illinois University and Syracuse University are public universities. #publicuniversity #Chicago #Syracuse</td>\n",- " <td id=\"T_c36cc_row1_col5\" class=\"data row1 col5\" >0.0</td>\n",+ " <th id=\"T_c3dbe_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",+ " <td id=\"T_c3dbe_row1_col0\" class=\"data row1 col0\" >Are both the University of Chicago and Syracuse University public universities? </td>\n",+ " <td id=\"T_c3dbe_row1_col1\" class=\"data row1 col1\" >no</td>\n",+ " <td id=\"T_c3dbe_row1_col2\" class=\"data row1 col2\" >{'Syracuse University', 'University of Chicago'}</td>\n",+ " <td id=\"T_c3dbe_row1_col3\" class=\"data row1 col3\" > Yes, both Northeastern Illinois University and Syracuse University are public universities. #publicuniversity #Chicago #Syracuse</td>\n",+ " <td id=\"T_c3dbe_row1_col4\" class=\"data row1 col4\" > Yes, both Northeastern Illinois University and Syracuse University are public universities. #publicuniversity #Chicago #Syracuse</td>\n",+ " <td id=\"T_c3dbe_row1_col5\" class=\"data row1 col5\" >0.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_c36cc_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",- " <td id=\"T_c36cc_row2_col0\" class=\"data row2 col0\" >In what region was Eddy Mazzoleni born?</td>\n",- " <td id=\"T_c36cc_row2_col1\" class=\"data row2 col1\" >Lombardy, northern Italy</td>\n",- " <td id=\"T_c36cc_row2_col2\" class=\"data row2 col2\" >{'Bergamo', 'Eddy Mazzoleni'}</td>\n",- " <td id=\"T_c36cc_row2_col3\" class=\"data row2 col3\" > Eddy Mazzoleni was born in Bergamo, a city in the Lombardy region of Italy. #EddyMazzoleni #Italy #Lombardy</td>\n",- " <td id=\"T_c36cc_row2_col4\" class=\"data row2 col4\" > Eddy Mazzoleni was born in Bergamo, a city in the Lombardy region of Italy. #EddyMazzoleni #Italy #Lombardy</td>\n",- " <td id=\"T_c36cc_row2_col5\" class=\"data row2 col5\" >1.0</td>\n",+ " <th id=\"T_c3dbe_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",+ " <td id=\"T_c3dbe_row2_col0\" class=\"data row2 col0\" >In what region was Eddy Mazzoleni born?</td>\n",+ " <td id=\"T_c3dbe_row2_col1\" class=\"data row2 col1\" >Lombardy, northern Italy</td>\n",+ " <td id=\"T_c3dbe_row2_col2\" class=\"data row2 col2\" >{'Bergamo', 'Eddy Mazzoleni'}</td>\n",+ " <td id=\"T_c3dbe_row2_col3\" class=\"data row2 col3\" > Eddy Mazzoleni was born in Bergamo, a city in the Lombardy region of Italy. #EddyMazzoleni #Italy #Lombardy</td>\n",+ " <td id=\"T_c3dbe_row2_col4\" class=\"data row2 col4\" > Eddy Mazzoleni was born in Bergamo, a city in the Lombardy region of Italy. #EddyMazzoleni #Italy #Lombardy</td>\n",+ " <td id=\"T_c3dbe_row2_col5\" class=\"data row2 col5\" >1.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_c36cc_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",- " <td id=\"T_c36cc_row3_col0\" class=\"data row3 col0\" >Who edited the 1990 American romantic comedy film directed by Garry Marshall?</td>\n",- " <td id=\"T_c36cc_row3_col1\" class=\"data row3 col1\" >Raja Raymond Gosnell</td>\n",- " <td id=\"T_c36cc_row3_col2\" class=\"data row3 col2\" >{'Raja Gosnell', 'Pretty Woman'}</td>\n",- " <td id=\"T_c36cc_row3_col3\" class=\"data row3 col3\" > Garry Marshall directed and edited the 1990 American romantic comedy film \"Pretty Woman\", starring Richard Gere and Julia Roberts. #PrettyWoman #GarryMarshall #RomanticComedy</td>\n",- " <td id=\"T_c36cc_row3_col4\" class=\"data row3 col4\" > Garry Marshall directed and edited the 1990 American romantic comedy film \"Pretty Woman\", starring Richard Gere and Julia Roberts. #PrettyWoman #GarryMarshall #RomanticComedy</td>\n",- " <td id=\"T_c36cc_row3_col5\" class=\"data row3 col5\" >0.0</td>\n",+ " <th id=\"T_c3dbe_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",+ " <td id=\"T_c3dbe_row3_col0\" class=\"data row3 col0\" >Who edited the 1990 American romantic comedy film directed by Garry Marshall?</td>\n",+ " <td id=\"T_c3dbe_row3_col1\" class=\"data row3 col1\" >Raja Raymond Gosnell</td>\n",+ " <td id=\"T_c3dbe_row3_col2\" class=\"data row3 col2\" >{'Raja Gosnell', 'Pretty Woman'}</td>\n",+ " <td id=\"T_c3dbe_row3_col3\" class=\"data row3 col3\" > Garry Marshall directed and edited the 1990 American romantic comedy film \"Pretty Woman\", starring Richard Gere and Julia Roberts. #PrettyWoman #GarryMarshall #RomanticComedy</td>\n",+ " <td id=\"T_c3dbe_row3_col4\" class=\"data row3 col4\" > Garry Marshall directed and edited the 1990 American romantic comedy film \"Pretty Woman\", starring Richard Gere and Julia Roberts. #PrettyWoman #GarryMarshall #RomanticComedy</td>\n",+ " <td id=\"T_c3dbe_row3_col5\" class=\"data row3 col5\" >0.0</td>\n"," </tr>\n"," <tr>\n",- " <th id=\"T_c36cc_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",- " <td id=\"T_c36cc_row4_col0\" class=\"data row4 col0\" >Burrs Country Park railway station is what stop on the railway line that runs between Heywood and Rawtenstall</td>\n",- " <td id=\"T_c36cc_row4_col1\" class=\"data row4 col1\" >seventh</td>\n",- " <td id=\"T_c36cc_row4_col2\" class=\"data row4 col2\" >{'East Lancashire Railway', 'Burrs Country Park railway station'}</td>\n",- " <td id=\"T_c36cc_row4_col3\" class=\"data row4 col3\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line, which runs between Heywood and Rawtenstall. #EastLancashireRailway #BurrsCountryPark #railwaystation</td>\n",- " <td id=\"T_c36cc_row4_col4\" class=\"data row4 col4\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line, which runs between Heywood and Rawtenstall. #EastLancashireRailway #BurrsCountryPark #railwaystation</td>\n",- " <td id=\"T_c36cc_row4_col5\" class=\"data row4 col5\" >1.0</td>\n",+ " <th id=\"T_c3dbe_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",+ " <td id=\"T_c3dbe_row4_col0\" class=\"data row4 col0\" >Burrs Country Park railway station is what stop on the railway line that runs between Heywood and Rawtenstall</td>\n",+ " <td id=\"T_c3dbe_row4_col1\" class=\"data row4 col1\" >seventh</td>\n",+ " <td id=\"T_c3dbe_row4_col2\" class=\"data row4 col2\" >{'East Lancashire Railway', 'Burrs Country Park railway station'}</td>\n",+ " <td id=\"T_c3dbe_row4_col3\" class=\"data row4 col3\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line, which runs between Heywood and Rawtenstall. #EastLancashireRailway #BurrsCountryPark #railwaystation</td>\n",+ " <td id=\"T_c3dbe_row4_col4\" class=\"data row4 col4\" > Burrs Country Park railway station is the seventh stop on the East Lancashire Railway line, which runs between Heywood and Rawtenstall. #EastLancashireRailway #BurrsCountryPark #railwaystation</td>\n",+ " <td id=\"T_c3dbe_row4_col5\" class=\"data row4 col5\" >1.0</td>\n"," </tr>\n"," </tbody>\n","</table>\n"],"text/plain": [- "<pandas.io.formats.style.Styler at 0x7f08b98dadc0>"+ "<pandas.io.formats.style.Styler at 0x7f8c608f90d0>"]},"metadata": {},@@ -879,15 +879,71 @@"metadata": {},"outputs": [{- "ename": "AttributeError",- "evalue": "'super' object has no attribute '__getattr__'",- "output_type": "error",- "traceback": [- "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",- "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",- "Cell \u001b[0;32mIn[12], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m prompt, output \u001b[38;5;241m=\u001b[39m \u001b[43mdspy\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msettings\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mlangchain_history\u001b[49m[\u001b[38;5;241m-\u001b[39m\u001b[38;5;241m1\u001b[39m]\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mPROMPT:\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m'\u001b[39m, prompt)\n\u001b[1;32m 4\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124mOUTPUT:\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124m'\u001b[39m, output)\n",- "File \u001b[0;32m/scr-ssd/okhattab/miniconda3/envs/py39_jan2024_01/lib/python3.9/site-packages/dsp/utils/settings.py:61\u001b[0m, in \u001b[0;36mSettings.__getattr__\u001b[0;34m(self, name)\u001b[0m\n\u001b[1;32m 58\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m name \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mconfig:\n\u001b[1;32m 59\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mconfig[name]\n\u001b[0;32m---> 61\u001b[0m \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[38;5;21;43m__getattr__\u001b[39;49m(name)\n",- "\u001b[0;31mAttributeError\u001b[0m: 'super' object has no attribute '__getattr__'"+ "name": "stdout",+ "output_type": "stream",+ "text": [+ "PROMPT:\n",+ "\n",+ " Essential Instructions: Respond to the provided question based on the given context in the style of a tweet, which typically requires a concise and engaging answer within the character limit of a tweet (280 characters).\n",+ "\n",+ "---\n",+ "\n",+ "Follow the following format.\n",+ "\n",+ "Context: ${context}\n",+ "Question: ${question}\n",+ "Tweet Response: ${tweet_response}\n",+ "\n",+ "---\n",+ "\n",+ "Context:\n",+ "[1] «Candace Kita | Kita's first role was as a news anchor in the 1991 movie \"Stealth Hunters\". Kita's first recurring television role was in Fox's \"Masked Rider\", from 1995 to 1996. She appeared as a series regular lead in all 40 episodes. Kita also portrayed a frantic stewardess in a music video directed by Mark Pellington for the British group, Catherine Wheel, titled, \"Waydown\" in 1995. In 1996, Kita also appeared in the film \"Barb Wire\" (1996) and guest starred on \"The Wayans Bros.\". She also guest starred in \"Miriam Teitelbaum: Homicide\" with \"Saturday Night Live\" alumni Nora Dunn, \"Wall To Wall Records\" with Jordan Bridges, \"Even Stevens\", \"Felicity\" with Keri Russell, \"V.I.P.\" with Pamela Anderson, \"Girlfriends\", \"The Sweet Spot\" with Bill Murray, and \"Movies at Our House\". She also had recurring roles on the FX spoof, \"Son of the Beach\" from 2001 to 2002, ABC-Family's \"Dance Fever\" and Oxygen Network's \"Running with Scissors\". Kita also appeared in the films \"Little Heroes\" (2002) and \"Rennie's Landing\" (2001).»\n",+ "[2] «Jilly Kitzinger | Jilly Kitzinger is a fictional character in the science fiction series \"Torchwood\", portrayed by American actress Lauren Ambrose. The character was promoted as one of five new main characters to join \"Torchwood\" in its fourth series, \"\" (2011), as part of a new co-production between \"Torchwood\"' s British network, BBC One, and its American financiers on US premium television network Starz. Ambrose appears in seven of the ten episodes, and is credited as a \"special guest star\" throughout. Whilst reaction to the serial was mixed, Ambrose' portrayal was often singled out by critics for particular praise and in 2012 she received a Saturn Award nomination for Best Supporting Actress on Television.»\n",+ "[3] «Candace Brown | Candace June Brown (born June 15, 1980) is an American actress and comedian best known for her work on shows such as \"Grey's Anatomy\", \"Desperate Housewives\", \"Head Case\", The \"Wizards Of Waverly Place\". In 2011, she joined the guest cast for \"Torchwood\"' s fourth series' \"\", airing on BBC One in the United Kingdom and premium television network Starz.»\n",+ "[4] «Candace Elaine | Candace Elaine is a Canadian actress who has become a naturalized American citizen. Born 1972 in Edmonton, Alberta, Canada, Elaine is an accomplished dancer, fashionista, and stage and film actor. She most recently appeared opposite Stone Cold Steve Austin, Michael Shanks, and Michael Jai White in the action feature \"Tactical Force\", playing the role of Ilya Kalashnikova.»\n",+ "[5] «Amy Steel | Amy Steel (born Alice Amy Steel; May 3, 1960) is an American film and television actress. She is best known for her roles as Ginny Field in \"Friday the 13th Part 2\" (1981) and Kit Graham in \"April Fool's Day\" (1986). She has starred in films such as \"Exposed\" (1983), \"Walk Like a Man\" (1987), \"What Ever Happened to Baby Jane? \" (1991), and \"Tales of Poe\" (2014). Steel has had numerous guest appearances on several television series, such as \"Family Ties\" (1983), \"The A-Team\" (1983), \"Quantum Leap\" (1990), and \"China Beach\" (1991), as well as a starring role in \"The Powers of Matthew Star\" (1982–83).»\n",+ "Question: which American actor was Candace Kita guest starred with\n",+ "Tweet Response: Candace Kita has guest starred with many American actors, including Nora Dunn, Jordan Bridges, Keri Russell, Pamela Anderson, and Bill Murray. #CandaceKita #gueststar #Americanactors\n",+ "\n",+ "---\n",+ "\n",+ "Context:\n",+ "[1] «The Victorians | The Victorians - Their Story In Pictures is a 2009 British documentary series which focuses on Victorian art and culture. The four-part series is written and presented by Jeremy Paxman and debuted on BBC One at 9:00pm on Sunday 15 February 2009.»\n",+ "[2] «Victorian (comics) | The Victorian is a 25-issue comic book series published by Penny-Farthing Press and starting in 1999. The brainchild of creator Trainor Houghton, the series included a number of notable script writers and illustrators, including Len Wein, Glen Orbik and Howard Chaykin.»\n",+ "[3] «The Great Victorian Collection | The Great Victorian Collection, published in 1975, is a novel by Northern Irish-Canadian writer Brian Moore. Set in Carmel, California, it tells the story of a man who dreams that the empty parking lot he can see from his hotel window has been transformed by the arrival of a collection of priceless Victoriana on display in a vast open-air market. When he awakes he finds that he can no longer distinguish the dream from reality.»\n",+ "[4] «Victorian People | Victorian People: A Reassessment of Persons and Themes, 1851-1867 is a book by the historian Asa Briggs originally published in 1955. It is part of a trilogy that also incorporates \"Victorian Cities\" and \"Victorian Things\".»\n",+ "[5] «The Caxtons | The Caxtons: A Family Picture is an 1849 Victorian novel by Edward Bulwer-Lytton that was popular in its time.»\n",+ "Question: The Victorians - Their Story In Pictures is a documentary series written by an author born in what year?\n",+ "Tweet Response: The Victorians - Their Story In Pictures is a 2009 British documentary series written and presented by Jeremy Paxman, who was born in 1950. #Victorian #documentary #JeremyPaxman\n",+ "\n",+ "---\n",+ "\n",+ "Context:\n",+ "[1] «Tae Kwon Do Times | Tae Kwon Do Times is a magazine devoted to the martial art of taekwondo, and is published in the United States of America. While the title suggests that it focuses on taekwondo exclusively, the magazine also covers other Korean martial arts. \"Tae Kwon Do Times\" has published articles by a wide range of authors, including He-Young Kimm, Thomas Kurz, Scott Shaw, and Mark Van Schuyver.»\n",+ "[2] «Kwon Tae-man | Kwon Tae-man (born 1941) was an early Korean hapkido practitioner and a pioneer of the art, first in Korea and then in the United States. He formed one of the earliest dojang's for hapkido in the United States in Torrance, California, and has been featured in many magazine articles promoting the art.»\n",+ "[3] «Hee Il Cho | Cho Hee Il (born October 13, 1940) is a prominent Korean-American master of taekwondo, holding the rank of 9th \"dan\" in the martial art. He has written 11 martial art books, produced 70 martial art training videos, and has appeared on more than 70 martial arts magazine covers. Cho won several national and international competitions as a taekwondo competitor, and has appeared in several films, including \"Fight to Win\", \"Best of the Best\", \"Bloodsport II\", and \"Bloodsport III\". He founded the Action International Martial Arts Association (AIMAA) in 1980, and is its President. Cho is a member of both \"Black Belt\" magazine's Hall of Fame and \"Tae Kwon Do Times\" magazine's Hall of Fame.»\n",+ "[4] «West Coast Magazine | West Coast Magazine (1987–1998). was a three times a year Scottish literary publication consisting of poetry, short fiction, articles, essays and reviews. Founding editors were Gordon Giles, Kenny MacKenzie and Joe Murray. The proof issue appeared in October 1987 and contained some articles and poems that did not appear in official issues. West Coast Magazine (WCM) was initially funded by East Glasgow Gear Project and Glasgow City Council; ultimately funded by the Scottish Arts Council.»\n",+ "[5] «Southwest Art | Southwest Art is a magazine published by F+W that specializes in fine art depicting artwork of the American Southwest.»\n",+ "Question: Which magazine has published articles by Scott Shaw, Tae Kwon Do Times or Southwest Art?\n",+ "Tweet Response: Tae Kwon Do Times has published articles by Scott Shaw, along with other notable authors in the martial arts world. #TaeKwonDo #MartialArts #Magazine\n",+ "\n",+ "---\n",+ "\n",+ "Context:\n",+ "[1] «Philip Markoff | Philip Haynes Markoff (February 12, 1986 – August 15, 2010) was an American medical student who was charged with the armed robbery and murder of Julissa Brisman in a Boston, Massachusetts, hotel on April 14, 2009, and two other armed robberies.»\n",+ "[2] «Antonia Brenner | Antonia Brenner, better known as Mother Antonia (Spanish: Madre Antonia ), (December 1, 1926 – October 17, 2013) was an American Roman Catholic Religious Sister and activist who chose to reside and care for inmates at the notorious maximum-security La Mesa Prison in Tijuana, Mexico. As a result of her work, she founded a new religious institute called the Eudist Servants of the 11th Hour.»\n",+ "[3] «Luzira Maximum Security Prison | Luzira Maximum Security Prison is a maximum security prison for both men and women in Uganda. As at July 2016, it is the only maximum security prison in the country and houses Uganda's death row inmates.»\n",+ "[4] «Pleasant Valley State Prison | Pleasant Valley State Prison (PVSP) is a 640 acres minimum-to-maximum security state prison in Coalinga, Fresno County, California. The facility has housed convicted murderers Sirhan Sirhan, Erik Menendez, X-Raided, and Hans Reiser, among others.»\n",+ "[5] «Jon-Adrian Velazquez | Jon-Adrian Velazquez is an inmate in the maximum security Sing-Sing prison in New York who is serving a 25-year sentence after being convicted of the 1998 murder of a retired police officer. His case garnered considerable attention from the media ten years after his conviction, due to a visit and support from Martin Sheen and a long-term investigation by Dateline NBC producer Dan Slepian.»\n",+ "Question: Which maximum security jail housed the killer of Julissa brisman?\n",+ "Tweet Response:\n",+ "\n",+ "\n",+ "OUTPUT:\n",+ "\n",+ " Prediction(\n",+ " tweet_response=' Philip Markoff, the killer of Julissa Brisman, was housed in the maximum security jail Pleasant Valley State Prison in California. #PhilipMarkoff #PleasantValleyStatePrison #maximumsecurity'\n",+ ")\n"]}],