Skip to content

armgit / dspy

Public Imported from GitHub
0

DSPy: The framework for programming—not prompting—language models

Python

Updates to notebook

Omar Khattab18906e97cac7
diff --git a/intro.ipynb b/intro.ipynb
index 3fe1ded7c..294f2acee 100644
--- a/intro.ipynb
+++ b/intro.ipynb
@@ -17,7 +17,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Set up\n",
+ "### Setting Up\n",
"\n",
"We'll start by setting up the language model (LM) and retrieval model (RM).\n",
"\n",
@@ -275,7 +275,7 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "100%|██████████| 13/13 [00:00<00:00, 115.31it/s]\n"
+ "100%|██████████| 13/13 [00:00<00:00, 354.31it/s]\n"
]
},
{
@@ -289,14 +289,14 @@
"data": {
"text/html": [
"<style type=\"text/css\">\n",
- "#T_e645f_ th {\n",
+ "#T_070f8_ th {\n",
" text-align: left;\n",
"}\n",
- "#T_e645f_ td {\n",
+ "#T_070f8_ td {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
- "<table id=\"T_e645f_\">\n",
+ "<table id=\"T_070f8_\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
@@ -308,101 +308,101 @@
" </thead>\n",
" <tbody>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
- " <td id=\"T_e645f_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
- " <td id=\"T_e645f_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
- " <td id=\"T_e645f_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
- " <td id=\"T_e645f_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
+ " <th id=\"T_070f8_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
+ " <td id=\"T_070f8_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
+ " <td id=\"T_070f8_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
+ " <td id=\"T_070f8_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
+ " <td id=\"T_070f8_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
- " <td id=\"T_e645f_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
- " <td id=\"T_e645f_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
- " <td id=\"T_e645f_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
- " <td id=\"T_e645f_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
+ " <th id=\"T_070f8_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
+ " <td id=\"T_070f8_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
+ " <td id=\"T_070f8_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
+ " <td id=\"T_070f8_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
+ " <td id=\"T_070f8_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
- " <td id=\"T_e645f_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
- " <td id=\"T_e645f_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
- " <td id=\"T_e645f_row2_col2\" class=\"data row2 col2\" >Melbourne, Australia</td>\n",
- " <td id=\"T_e645f_row2_col3\" class=\"data row2 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
+ " <td id=\"T_070f8_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
+ " <td id=\"T_070f8_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
+ " <td id=\"T_070f8_row2_col2\" class=\"data row2 col2\" >Melbourne, Australia</td>\n",
+ " <td id=\"T_070f8_row2_col3\" class=\"data row2 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
- " <td id=\"T_e645f_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
- " <td id=\"T_e645f_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
- " <td id=\"T_e645f_row3_col2\" class=\"data row3 col2\" >1966</td>\n",
- " <td id=\"T_e645f_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
+ " <td id=\"T_070f8_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
+ " <td id=\"T_070f8_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
+ " <td id=\"T_070f8_row3_col2\" class=\"data row3 col2\" >1966</td>\n",
+ " <td id=\"T_070f8_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
- " <td id=\"T_e645f_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
- " <td id=\"T_e645f_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
- " <td id=\"T_e645f_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
- " <td id=\"T_e645f_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
+ " <th id=\"T_070f8_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
+ " <td id=\"T_070f8_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
+ " <td id=\"T_070f8_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
+ " <td id=\"T_070f8_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
+ " <td id=\"T_070f8_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
- " <td id=\"T_e645f_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
- " <td id=\"T_e645f_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
- " <td id=\"T_e645f_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
- " <td id=\"T_e645f_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
+ " <th id=\"T_070f8_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
+ " <td id=\"T_070f8_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
+ " <td id=\"T_070f8_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
+ " <td id=\"T_070f8_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
+ " <td id=\"T_070f8_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
- " <td id=\"T_e645f_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
- " <td id=\"T_e645f_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
- " <td id=\"T_e645f_row6_col2\" class=\"data row6 col2\" >Manchester, England</td>\n",
- " <td id=\"T_e645f_row6_col3\" class=\"data row6 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
+ " <td id=\"T_070f8_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
+ " <td id=\"T_070f8_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
+ " <td id=\"T_070f8_row6_col2\" class=\"data row6 col2\" >Manchester, England</td>\n",
+ " <td id=\"T_070f8_row6_col3\" class=\"data row6 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
- " <td id=\"T_e645f_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
- " <td id=\"T_e645f_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
- " <td id=\"T_e645f_row7_col2\" class=\"data row7 col2\" >July 3, 1961</td>\n",
- " <td id=\"T_e645f_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
+ " <td id=\"T_070f8_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
+ " <td id=\"T_070f8_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
+ " <td id=\"T_070f8_row7_col2\" class=\"data row7 col2\" >July 3, 1961</td>\n",
+ " <td id=\"T_070f8_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
- " <td id=\"T_e645f_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
- " <td id=\"T_e645f_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
- " <td id=\"T_e645f_row8_col2\" class=\"data row8 col2\" >Susquehanna River</td>\n",
- " <td id=\"T_e645f_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
+ " <td id=\"T_070f8_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
+ " <td id=\"T_070f8_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
+ " <td id=\"T_070f8_row8_col2\" class=\"data row8 col2\" >Susquehanna River</td>\n",
+ " <td id=\"T_070f8_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
- " <td id=\"T_e645f_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
- " <td id=\"T_e645f_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
- " <td id=\"T_e645f_row9_col2\" class=\"data row9 col2\" >Mars exploration</td>\n",
- " <td id=\"T_e645f_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
+ " <td id=\"T_070f8_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
+ " <td id=\"T_070f8_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
+ " <td id=\"T_070f8_row9_col2\" class=\"data row9 col2\" >Mars exploration</td>\n",
+ " <td id=\"T_070f8_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
- " <td id=\"T_e645f_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
- " <td id=\"T_e645f_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
- " <td id=\"T_e645f_row10_col2\" class=\"data row10 col2\" >Hong Kong</td>\n",
- " <td id=\"T_e645f_row10_col3\" class=\"data row10 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
+ " <td id=\"T_070f8_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
+ " <td id=\"T_070f8_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
+ " <td id=\"T_070f8_row10_col2\" class=\"data row10 col2\" >Hong Kong</td>\n",
+ " <td id=\"T_070f8_row10_col3\" class=\"data row10 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
- " <td id=\"T_e645f_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
- " <td id=\"T_e645f_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
- " <td id=\"T_e645f_row11_col2\" class=\"data row11 col2\" >Venus</td>\n",
- " <td id=\"T_e645f_row11_col3\" class=\"data row11 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
+ " <td id=\"T_070f8_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
+ " <td id=\"T_070f8_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
+ " <td id=\"T_070f8_row11_col2\" class=\"data row11 col2\" >Venus</td>\n",
+ " <td id=\"T_070f8_row11_col3\" class=\"data row11 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_e645f_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
- " <td id=\"T_e645f_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
- " <td id=\"T_e645f_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
- " <td id=\"T_e645f_row12_col2\" class=\"data row12 col2\" >1457</td>\n",
- " <td id=\"T_e645f_row12_col3\" class=\"data row12 col3\" >❌</td>\n",
+ " <th id=\"T_070f8_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
+ " <td id=\"T_070f8_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
+ " <td id=\"T_070f8_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
+ " <td id=\"T_070f8_row12_col2\" class=\"data row12 col2\" >1457</td>\n",
+ " <td id=\"T_070f8_row12_col3\" class=\"data row12 col3\" >❌</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
- "<pandas.io.formats.style.Styler at 0x7f0914ef0250>"
+ "<pandas.io.formats.style.Styler at 0x7fa029297fd0>"
]
},
"metadata": {},
@@ -572,7 +572,7 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "100%|██████████| 13/13 [00:00<00:00, 63.50it/s]"
+ "100%|██████████| 13/13 [00:00<00:00, 244.05it/s]"
]
},
{
@@ -593,14 +593,14 @@
"data": {
"text/html": [
"<style type=\"text/css\">\n",
- "#T_f97da_ th {\n",
+ "#T_51a84_ th {\n",
" text-align: left;\n",
"}\n",
- "#T_f97da_ td {\n",
+ "#T_51a84_ td {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
- "<table id=\"T_f97da_\">\n",
+ "<table id=\"T_51a84_\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
@@ -612,101 +612,101 @@
" </thead>\n",
" <tbody>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
- " <td id=\"T_f97da_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
- " <td id=\"T_f97da_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
- " <td id=\"T_f97da_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
- " <td id=\"T_f97da_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
+ " <th id=\"T_51a84_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
+ " <td id=\"T_51a84_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
+ " <td id=\"T_51a84_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
+ " <td id=\"T_51a84_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
+ " <td id=\"T_51a84_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
- " <td id=\"T_f97da_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
- " <td id=\"T_f97da_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
- " <td id=\"T_f97da_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
- " <td id=\"T_f97da_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
+ " <th id=\"T_51a84_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
+ " <td id=\"T_51a84_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
+ " <td id=\"T_51a84_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
+ " <td id=\"T_51a84_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
+ " <td id=\"T_51a84_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
- " <td id=\"T_f97da_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
- " <td id=\"T_f97da_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
- " <td id=\"T_f97da_row2_col2\" class=\"data row2 col2\" >Daytona Beach, Florida</td>\n",
- " <td id=\"T_f97da_row2_col3\" class=\"data row2 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
+ " <td id=\"T_51a84_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
+ " <td id=\"T_51a84_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
+ " <td id=\"T_51a84_row2_col2\" class=\"data row2 col2\" >Daytona Beach, Florida</td>\n",
+ " <td id=\"T_51a84_row2_col3\" class=\"data row2 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
- " <td id=\"T_f97da_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
- " <td id=\"T_f97da_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
- " <td id=\"T_f97da_row3_col2\" class=\"data row3 col2\" >1856</td>\n",
- " <td id=\"T_f97da_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
+ " <td id=\"T_51a84_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
+ " <td id=\"T_51a84_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
+ " <td id=\"T_51a84_row3_col2\" class=\"data row3 col2\" >1856</td>\n",
+ " <td id=\"T_51a84_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
- " <td id=\"T_f97da_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
- " <td id=\"T_f97da_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
- " <td id=\"T_f97da_row4_col2\" class=\"data row4 col2\" >Studs Terkel</td>\n",
- " <td id=\"T_f97da_row4_col3\" class=\"data row4 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
+ " <td id=\"T_51a84_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
+ " <td id=\"T_51a84_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
+ " <td id=\"T_51a84_row4_col2\" class=\"data row4 col2\" >Studs Terkel</td>\n",
+ " <td id=\"T_51a84_row4_col3\" class=\"data row4 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
- " <td id=\"T_f97da_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
- " <td id=\"T_f97da_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
- " <td id=\"T_f97da_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
- " <td id=\"T_f97da_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
+ " <th id=\"T_51a84_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
+ " <td id=\"T_51a84_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
+ " <td id=\"T_51a84_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
+ " <td id=\"T_51a84_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
+ " <td id=\"T_51a84_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
- " <td id=\"T_f97da_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
- " <td id=\"T_f97da_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
- " <td id=\"T_f97da_row6_col2\" class=\"data row6 col2\" >Manchester, England</td>\n",
- " <td id=\"T_f97da_row6_col3\" class=\"data row6 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
+ " <td id=\"T_51a84_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
+ " <td id=\"T_51a84_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
+ " <td id=\"T_51a84_row6_col2\" class=\"data row6 col2\" >Manchester, England</td>\n",
+ " <td id=\"T_51a84_row6_col3\" class=\"data row6 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
- " <td id=\"T_f97da_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
- " <td id=\"T_f97da_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
- " <td id=\"T_f97da_row7_col2\" class=\"data row7 col2\" >5 August 1862</td>\n",
- " <td id=\"T_f97da_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
+ " <td id=\"T_51a84_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
+ " <td id=\"T_51a84_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
+ " <td id=\"T_51a84_row7_col2\" class=\"data row7 col2\" >5 August 1862</td>\n",
+ " <td id=\"T_51a84_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
- " <td id=\"T_f97da_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
- " <td id=\"T_f97da_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
- " <td id=\"T_f97da_row8_col2\" class=\"data row8 col2\" >Dundalk</td>\n",
- " <td id=\"T_f97da_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
+ " <td id=\"T_51a84_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
+ " <td id=\"T_51a84_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
+ " <td id=\"T_51a84_row8_col2\" class=\"data row8 col2\" >Dundalk</td>\n",
+ " <td id=\"T_51a84_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
- " <td id=\"T_f97da_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
- " <td id=\"T_f97da_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
- " <td id=\"T_f97da_row9_col2\" class=\"data row9 col2\" >Jet Propulsion</td>\n",
- " <td id=\"T_f97da_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
+ " <td id=\"T_51a84_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
+ " <td id=\"T_51a84_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
+ " <td id=\"T_51a84_row9_col2\" class=\"data row9 col2\" >Jet Propulsion</td>\n",
+ " <td id=\"T_51a84_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
- " <td id=\"T_f97da_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
- " <td id=\"T_f97da_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
- " <td id=\"T_f97da_row10_col2\" class=\"data row10 col2\" >Los Angeles, California</td>\n",
- " <td id=\"T_f97da_row10_col3\" class=\"data row10 col3\" >❌</td>\n",
+ " <th id=\"T_51a84_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
+ " <td id=\"T_51a84_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
+ " <td id=\"T_51a84_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
+ " <td id=\"T_51a84_row10_col2\" class=\"data row10 col2\" >Los Angeles, California</td>\n",
+ " <td id=\"T_51a84_row10_col3\" class=\"data row10 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
- " <td id=\"T_f97da_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
- " <td id=\"T_f97da_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
- " <td id=\"T_f97da_row11_col2\" class=\"data row11 col2\" >Portrait of a Marriage</td>\n",
- " <td id=\"T_f97da_row11_col3\" class=\"data row11 col3\" >✔️</td>\n",
+ " <th id=\"T_51a84_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
+ " <td id=\"T_51a84_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
+ " <td id=\"T_51a84_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
+ " <td id=\"T_51a84_row11_col2\" class=\"data row11 col2\" >Portrait of a Marriage</td>\n",
+ " <td id=\"T_51a84_row11_col3\" class=\"data row11 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_f97da_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
- " <td id=\"T_f97da_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
- " <td id=\"T_f97da_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
- " <td id=\"T_f97da_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
- " <td id=\"T_f97da_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
+ " <th id=\"T_51a84_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
+ " <td id=\"T_51a84_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
+ " <td id=\"T_51a84_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
+ " <td id=\"T_51a84_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
+ " <td id=\"T_51a84_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
- "<pandas.io.formats.style.Styler at 0x7f090f1b0450>"
+ "<pandas.io.formats.style.Styler at 0x7fa01eef6d90>"
]
},
"metadata": {},
@@ -895,7 +895,7 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "100%|██████████| 13/13 [00:01<00:00, 8.49it/s]"
+ "100%|██████████| 13/13 [00:00<00:00, 18.90it/s]"
]
},
{
@@ -916,14 +916,14 @@
"data": {
"text/html": [
"<style type=\"text/css\">\n",
- "#T_52677_ th {\n",
+ "#T_2cf39_ th {\n",
" text-align: left;\n",
"}\n",
- "#T_52677_ td {\n",
+ "#T_2cf39_ td {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
- "<table id=\"T_52677_\">\n",
+ "<table id=\"T_2cf39_\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
@@ -935,101 +935,101 @@
" </thead>\n",
" <tbody>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
- " <td id=\"T_52677_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
- " <td id=\"T_52677_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
- " <td id=\"T_52677_row0_col2\" class=\"data row0 col2\" >Julia Peterkin</td>\n",
- " <td id=\"T_52677_row0_col3\" class=\"data row0 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
+ " <td id=\"T_2cf39_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
+ " <td id=\"T_2cf39_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
+ " <td id=\"T_2cf39_row0_col2\" class=\"data row0 col2\" >Julia Peterkin</td>\n",
+ " <td id=\"T_2cf39_row0_col3\" class=\"data row0 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
- " <td id=\"T_52677_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
- " <td id=\"T_52677_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
- " <td id=\"T_52677_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
- " <td id=\"T_52677_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
+ " <th id=\"T_2cf39_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
+ " <td id=\"T_2cf39_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
+ " <td id=\"T_2cf39_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
+ " <td id=\"T_2cf39_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
+ " <td id=\"T_2cf39_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
- " <td id=\"T_52677_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
- " <td id=\"T_52677_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
- " <td id=\"T_52677_row2_col2\" class=\"data row2 col2\" >Ocala, Florida</td>\n",
- " <td id=\"T_52677_row2_col3\" class=\"data row2 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
+ " <td id=\"T_2cf39_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
+ " <td id=\"T_2cf39_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
+ " <td id=\"T_2cf39_row2_col2\" class=\"data row2 col2\" >Ocala, Florida</td>\n",
+ " <td id=\"T_2cf39_row2_col3\" class=\"data row2 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
- " <td id=\"T_52677_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
- " <td id=\"T_52677_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
- " <td id=\"T_52677_row3_col2\" class=\"data row3 col2\" >1824</td>\n",
- " <td id=\"T_52677_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
+ " <td id=\"T_2cf39_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
+ " <td id=\"T_2cf39_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
+ " <td id=\"T_2cf39_row3_col2\" class=\"data row3 col2\" >1824</td>\n",
+ " <td id=\"T_2cf39_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
- " <td id=\"T_52677_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
- " <td id=\"T_52677_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
- " <td id=\"T_52677_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
- " <td id=\"T_52677_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
+ " <th id=\"T_2cf39_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
+ " <td id=\"T_2cf39_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
+ " <td id=\"T_2cf39_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
+ " <td id=\"T_2cf39_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
+ " <td id=\"T_2cf39_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
- " <td id=\"T_52677_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
- " <td id=\"T_52677_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
- " <td id=\"T_52677_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
- " <td id=\"T_52677_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
+ " <th id=\"T_2cf39_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
+ " <td id=\"T_2cf39_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
+ " <td id=\"T_2cf39_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
+ " <td id=\"T_2cf39_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
+ " <td id=\"T_2cf39_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
- " <td id=\"T_52677_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
- " <td id=\"T_52677_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
- " <td id=\"T_52677_row6_col2\" class=\"data row6 col2\" >Cheshire, UK</td>\n",
- " <td id=\"T_52677_row6_col3\" class=\"data row6 col3\" >✔️</td>\n",
+ " <th id=\"T_2cf39_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
+ " <td id=\"T_2cf39_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
+ " <td id=\"T_2cf39_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
+ " <td id=\"T_2cf39_row6_col2\" class=\"data row6 col2\" >Cheshire, UK</td>\n",
+ " <td id=\"T_2cf39_row6_col3\" class=\"data row6 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
- " <td id=\"T_52677_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
- " <td id=\"T_52677_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
- " <td id=\"T_52677_row7_col2\" class=\"data row7 col2\" >September 27, 1974</td>\n",
- " <td id=\"T_52677_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
+ " <td id=\"T_2cf39_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
+ " <td id=\"T_2cf39_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
+ " <td id=\"T_2cf39_row7_col2\" class=\"data row7 col2\" >September 27, 1974</td>\n",
+ " <td id=\"T_2cf39_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
- " <td id=\"T_52677_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
- " <td id=\"T_52677_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
- " <td id=\"T_52677_row8_col2\" class=\"data row8 col2\" >Dundalk</td>\n",
- " <td id=\"T_52677_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
+ " <td id=\"T_2cf39_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
+ " <td id=\"T_2cf39_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
+ " <td id=\"T_2cf39_row8_col2\" class=\"data row8 col2\" >Dundalk</td>\n",
+ " <td id=\"T_2cf39_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
- " <td id=\"T_52677_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
- " <td id=\"T_52677_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
- " <td id=\"T_52677_row9_col2\" class=\"data row9 col2\" >Aerospace engineering</td>\n",
- " <td id=\"T_52677_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
+ " <td id=\"T_2cf39_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
+ " <td id=\"T_2cf39_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
+ " <td id=\"T_2cf39_row9_col2\" class=\"data row9 col2\" >Aerospace engineering</td>\n",
+ " <td id=\"T_2cf39_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
- " <td id=\"T_52677_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
- " <td id=\"T_52677_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
- " <td id=\"T_52677_row10_col2\" class=\"data row10 col2\" >Italy</td>\n",
- " <td id=\"T_52677_row10_col3\" class=\"data row10 col3\" >❌</td>\n",
+ " <th id=\"T_2cf39_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
+ " <td id=\"T_2cf39_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
+ " <td id=\"T_2cf39_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
+ " <td id=\"T_2cf39_row10_col2\" class=\"data row10 col2\" >Italy</td>\n",
+ " <td id=\"T_2cf39_row10_col3\" class=\"data row10 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
- " <td id=\"T_52677_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
- " <td id=\"T_52677_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
- " <td id=\"T_52677_row11_col2\" class=\"data row11 col2\" >Portrait of a Marriage</td>\n",
- " <td id=\"T_52677_row11_col3\" class=\"data row11 col3\" >✔️</td>\n",
+ " <th id=\"T_2cf39_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
+ " <td id=\"T_2cf39_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
+ " <td id=\"T_2cf39_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
+ " <td id=\"T_2cf39_row11_col2\" class=\"data row11 col2\" >Portrait of a Marriage</td>\n",
+ " <td id=\"T_2cf39_row11_col3\" class=\"data row11 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_52677_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
- " <td id=\"T_52677_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
- " <td id=\"T_52677_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
- " <td id=\"T_52677_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
- " <td id=\"T_52677_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
+ " <th id=\"T_2cf39_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
+ " <td id=\"T_2cf39_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
+ " <td id=\"T_2cf39_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
+ " <td id=\"T_2cf39_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
+ " <td id=\"T_2cf39_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
- "<pandas.io.formats.style.Styler at 0x7f0915703d50>"
+ "<pandas.io.formats.style.Styler at 0x7fa01e93e3d0>"
]
},
"metadata": {},
@@ -1318,7 +1318,7 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "100%|██████████| 13/13 [00:00<00:00, 23.89it/s]"
+ "100%|██████████| 13/13 [00:00<00:00, 89.68it/s]"
]
},
{
@@ -1339,14 +1339,14 @@
"data": {
"text/html": [
"<style type=\"text/css\">\n",
- "#T_2baa7_ th {\n",
+ "#T_f09c3_ th {\n",
" text-align: left;\n",
"}\n",
- "#T_2baa7_ td {\n",
+ "#T_f09c3_ td {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
- "<table id=\"T_2baa7_\">\n",
+ "<table id=\"T_f09c3_\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
@@ -1358,101 +1358,101 @@
" </thead>\n",
" <tbody>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
- " <td id=\"T_2baa7_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
- " <td id=\"T_2baa7_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
- " <td id=\"T_2baa7_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
- " <td id=\"T_2baa7_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
+ " <td id=\"T_f09c3_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
+ " <td id=\"T_f09c3_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
+ " <td id=\"T_f09c3_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
+ " <td id=\"T_f09c3_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
- " <td id=\"T_2baa7_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
- " <td id=\"T_2baa7_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
- " <td id=\"T_2baa7_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
- " <td id=\"T_2baa7_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
+ " <td id=\"T_f09c3_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
+ " <td id=\"T_f09c3_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
+ " <td id=\"T_f09c3_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
+ " <td id=\"T_f09c3_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
- " <td id=\"T_2baa7_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
- " <td id=\"T_2baa7_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
- " <td id=\"T_2baa7_row2_col2\" class=\"data row2 col2\" >Gainesville, Florida</td>\n",
- " <td id=\"T_2baa7_row2_col3\" class=\"data row2 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
+ " <td id=\"T_f09c3_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
+ " <td id=\"T_f09c3_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
+ " <td id=\"T_f09c3_row2_col2\" class=\"data row2 col2\" >Gainesville, Florida</td>\n",
+ " <td id=\"T_f09c3_row2_col3\" class=\"data row2 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
- " <td id=\"T_2baa7_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
- " <td id=\"T_2baa7_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
- " <td id=\"T_2baa7_row3_col2\" class=\"data row3 col2\" >1947</td>\n",
- " <td id=\"T_2baa7_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
+ " <th id=\"T_f09c3_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
+ " <td id=\"T_f09c3_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
+ " <td id=\"T_f09c3_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
+ " <td id=\"T_f09c3_row3_col2\" class=\"data row3 col2\" >1947</td>\n",
+ " <td id=\"T_f09c3_row3_col3\" class=\"data row3 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
- " <td id=\"T_2baa7_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
- " <td id=\"T_2baa7_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
- " <td id=\"T_2baa7_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
- " <td id=\"T_2baa7_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
+ " <td id=\"T_f09c3_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
+ " <td id=\"T_f09c3_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
+ " <td id=\"T_f09c3_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
+ " <td id=\"T_f09c3_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
- " <td id=\"T_2baa7_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
- " <td id=\"T_2baa7_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
- " <td id=\"T_2baa7_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
- " <td id=\"T_2baa7_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
+ " <td id=\"T_f09c3_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
+ " <td id=\"T_f09c3_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
+ " <td id=\"T_f09c3_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
+ " <td id=\"T_f09c3_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
- " <td id=\"T_2baa7_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
- " <td id=\"T_2baa7_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
- " <td id=\"T_2baa7_row6_col2\" class=\"data row6 col2\" >Manchester, England</td>\n",
- " <td id=\"T_2baa7_row6_col3\" class=\"data row6 col3\" >❌</td>\n",
+ " <th id=\"T_f09c3_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
+ " <td id=\"T_f09c3_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
+ " <td id=\"T_f09c3_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
+ " <td id=\"T_f09c3_row6_col2\" class=\"data row6 col2\" >Manchester, England</td>\n",
+ " <td id=\"T_f09c3_row6_col3\" class=\"data row6 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
- " <td id=\"T_2baa7_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
- " <td id=\"T_2baa7_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
- " <td id=\"T_2baa7_row7_col2\" class=\"data row7 col2\" >1965</td>\n",
- " <td id=\"T_2baa7_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
+ " <th id=\"T_f09c3_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
+ " <td id=\"T_f09c3_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
+ " <td id=\"T_f09c3_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
+ " <td id=\"T_f09c3_row7_col2\" class=\"data row7 col2\" >1965</td>\n",
+ " <td id=\"T_f09c3_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
- " <td id=\"T_2baa7_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
- " <td id=\"T_2baa7_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
- " <td id=\"T_2baa7_row8_col2\" class=\"data row8 col2\" >River Foyle</td>\n",
- " <td id=\"T_2baa7_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
+ " <th id=\"T_f09c3_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
+ " <td id=\"T_f09c3_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
+ " <td id=\"T_f09c3_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
+ " <td id=\"T_f09c3_row8_col2\" class=\"data row8 col2\" >River Foyle</td>\n",
+ " <td id=\"T_f09c3_row8_col3\" class=\"data row8 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
- " <td id=\"T_2baa7_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
- " <td id=\"T_2baa7_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
- " <td id=\"T_2baa7_row9_col2\" class=\"data row9 col2\" >Aerospace engineering</td>\n",
- " <td id=\"T_2baa7_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
+ " <th id=\"T_f09c3_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
+ " <td id=\"T_f09c3_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
+ " <td id=\"T_f09c3_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
+ " <td id=\"T_f09c3_row9_col2\" class=\"data row9 col2\" >Aerospace engineering</td>\n",
+ " <td id=\"T_f09c3_row9_col3\" class=\"data row9 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
- " <td id=\"T_2baa7_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
- " <td id=\"T_2baa7_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
- " <td id=\"T_2baa7_row10_col2\" class=\"data row10 col2\" >Guildford, Surrey</td>\n",
- " <td id=\"T_2baa7_row10_col3\" class=\"data row10 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
+ " <td id=\"T_f09c3_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
+ " <td id=\"T_f09c3_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
+ " <td id=\"T_f09c3_row10_col2\" class=\"data row10 col2\" >Guildford, Surrey</td>\n",
+ " <td id=\"T_f09c3_row10_col3\" class=\"data row10 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
- " <td id=\"T_2baa7_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
- " <td id=\"T_2baa7_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
- " <td id=\"T_2baa7_row11_col2\" class=\"data row11 col2\" >Carrington</td>\n",
- " <td id=\"T_2baa7_row11_col3\" class=\"data row11 col3\" >❌</td>\n",
+ " <th id=\"T_f09c3_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
+ " <td id=\"T_f09c3_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
+ " <td id=\"T_f09c3_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
+ " <td id=\"T_f09c3_row11_col2\" class=\"data row11 col2\" >Carrington</td>\n",
+ " <td id=\"T_f09c3_row11_col3\" class=\"data row11 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_2baa7_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
- " <td id=\"T_2baa7_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
- " <td id=\"T_2baa7_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
- " <td id=\"T_2baa7_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
- " <td id=\"T_2baa7_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
+ " <th id=\"T_f09c3_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
+ " <td id=\"T_f09c3_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
+ " <td id=\"T_f09c3_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
+ " <td id=\"T_f09c3_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
+ " <td id=\"T_f09c3_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
- "<pandas.io.formats.style.Styler at 0x7f090de2b5d0>"
+ "<pandas.io.formats.style.Styler at 0x7fa01d1be810>"
]
},
"metadata": {},
@@ -1482,7 +1482,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "We address these problems automatically in the program below. In it, we begin by defining `multihop_demonstrate` (which uses `multihop_attempt`) to automatically **annotate** demonstrations for the complex multi-hop pipeline. These demonstrations will be provided to the LM when it's invoked for each transformation."
+ "We address these problems automatically in the program below.\n",
+ "\n",
+ "In it, we begin by defining `multihop_demonstrate` (which uses `multihop_attempt`) to automatically **annotate** demonstrations for the complex multi-hop pipeline. These demonstrations will be provided to the LM when it's invoked for each transformation."
]
},
{
@@ -1514,6 +1516,14 @@
" return x"
]
},
+ {
+ "attachments": {},
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We now implement `multihop_search_v2` as part of `multihop_QA_v2`. In addition to the changes mentioned earlier, this program simulates the Baleen system (Khattab et al., 2021) in a few lines of code. In each retrieval hop (after the very first hop), the summary of the previous hop(s) is included in the prompt. This allows us to efficiently read a larger number of passages from the current hop."
+ ]
+ },
{
"cell_type": "code",
"execution_count": 23,
@@ -1566,84 +1576,41 @@
"\n",
"Follow the following format.\n",
"\n",
- "Context:\n",
- "${one or more sources, some of which may contain relevant content or clues}\n",
- "\n",
"Question: ${the question to be answered}\n",
- "\n",
- "Rationale: Let's think step by step. We now know that ${information from the context that may provide useful clues for the question}\n",
- "\n",
+ "Rationale: Let's think step by step. To answer this question, we first need to find out ${missing information}\n",
"Search Query: ${a simple question for seeking the missing or required information}\n",
"\n",
"---\n",
"\n",
- "Context:\n",
- "[1] «Gary Zukav | Gary Zukav Gary Zukav (born October 17, 1942) is an American spiritual teacher and the author of four consecutive New York Times Best Sellers. Beginning in 1998, he appeared more than 30 times on \"The Oprah Winfrey Show\" to discuss transformation in human consciousness concepts presented in his book \"The Seat of the Soul\". His first book, \"The Dancing Wu Li Masters\" (1979), won a U.S. National Book Award. Gary Zukav was born in Port Arthur, Texas, and spent his early childhood in San Antonio and Houston. His family moved to Pittsburg, Kansas, while he was in fourth grade. In»\n",
- "[2] «Gary Zukav | experience that led to his first book, \"Dancing Wu Li Masters\", written with extensive help from Jack Sarfatti and other physicists he met through Sarfatti as described in David Kaiser's book \"How the Hippies Saved Physics\". He later described this book as his \"first gift to Life\". In 1987 he moved to Mount Shasta, California, where he lived in a cabin as a self-described \"secular monk\" and spent extensive time in the surrounding wilderness. In 1993 he met Linda Francis. They co-founded the Seat of the Soul Institute in 1998 and moved to Ashland, Oregon, in 2000. Christopher Lehmann-Haupt reviewed»\n",
- "\n",
"Question: Which award did the first book of Gary Zukav receive?\n",
- "\n",
- "Rationale: Let's think step by step. We now know that Gary Zukav's first book is \"The Dancing Wu Li Masters\" and it was written with the help of Jack Sarfatti and other physicists.\n",
- "\n",
- "Search Query: What award did \"The Dancing Wu Li Masters\" receive?\n",
+ "Rationale: Let's think step by step. To answer this question, we first need to find out the name of the first book of Gary Zukav\n",
+ "Search Query: \"Gary Zukav\" first book\n",
"\n",
"---\n",
"\n",
- "Context:\n",
- "[1] «John du Pont | John du Pont John Eleuthère du Pont (November 22, 1938 – December 9, 2010) was an American philanthropist, heir to the Du Pont family fortune, and a convicted murderer. He had been a published ornithologist, philatelist, conchologist, sports enthusiast, and self-styled wrestling coach. He died in prison while serving a sentence of 30 years for the murder of Dave Schultz. In 1972, du Pont founded and directed the Delaware Museum of Natural History and contributed to Villanova University and other institutions. In the 1980s, he established a wrestling facility at his Foxcatcher Farm after becoming interested in the sport and»\n",
- "[2] «Du Pont family | with political and business ventures, as well as philanthropic efforts on behalf of various organizations such as The Jockey Club, the Philadelphia Orchestra, the Delaware Art Museum, and the Grand Opera House. Two family members were the subjects of well-publicized criminal cases. Heir John Eleuthère du Pont was convicted of murdering wrestling coach Dave Schultz in 1996 and heir Robert H. Richards IV was convicted of raping his 3-year-old daughter in 2009. The family is depicted in the 2014 biographical film \"Foxcatcher\" with Steve Carell playing John Eleuthère du Pont and Vanessa Redgrave playing Jean du Pont, the wife of»\n",
- "\n",
"Question: The heir to the Du Pont family fortune sponsored what wrestling team?\n",
- "\n",
- "Rationale: Let's think step by step. We now know that John du Pont was an American philanthropist and heir to the Du Pont family fortune. He was also a published ornithologist, philatelist, conchologist, sports enthusiast, and self-styled wrestling coach.\n",
- "\n",
- "Search Query: What wrestling team did John du Pont sponsor?\n",
+ "Rationale: Let's think step by step. To answer this question, we first need to find out who the heir to the Du Pont family fortune is.\n",
+ "Search Query: \"Heir to the Du Pont family fortune\"\n",
"\n",
"---\n",
"\n",
- "Context:\n",
- "[1] «Peter Outerbridge | Peter Outerbridge Peter Outerbridge (born June 30, 1966) is a Canadian actor, best known for his role as Ari Tasarov in the CW action series \"Nikita\", Dr. David Sandström in the TMN series \"ReGenesis\", Henrik \"Hank\" Johanssen in \"Orphan Black\", Bob Corbett in \"Bomb Girls\", William Easton in \"Saw VI\" and George Brown in the television film \"\". He also played the lead role of Detective William Murdoch in a three-episode mini-series, \"The Murdoch Mysteries\", in its initial run on Canadian television, with two episodes shown in 2004 and a third in 2005. Outerbridge was born and raised in Toronto,»\n",
- "[2] «Except the Dying | 2008. On March 3, 2015, Acorn Media announced a re-release for all three movies, set for May 26, 2015. Except the Dying Except the Dying is a 2004 made-for-TV film starring Peter Outerbridge, Colm Meaney, Keeley Hawes, William B. Davis and Flora Montgomery. It was adapted by Janet MacLean from the novel of the same name by Maureen Jennings. In this film, William Murdoch is introduced, as a man of strong principles, who uses his unique abilities to solve crimes, sometimes using advanced science for his time. On the street of Toronto, in the 1890s, the naked body of a»\n",
- "\n",
"Question: Who was the director of the 2009 movie featuring Peter Outerbridge as William Easton?\n",
- "\n",
- "Rationale: Let's think step by step. We now know that Peter Outerbridge played William Easton in the 2009 movie.\n",
- "\n",
- "Search Query: Who was the director of the 2009 movie featuring Peter Outerbridge as William Easton?\n",
+ "Rationale: Let's think step by step. To answer this question, we first need to find out the name of the 2009 movie featuring Peter Outerbridge as William Easton.\n",
+ "Search Query: \"2009 movie featuring Peter Outerbridge as William Easton\"\n",
"\n",
"---\n",
"\n",
- "Context:\n",
- "[1] «Julia Peterkin | Julia Peterkin Julia Peterkin (October 31, 1880 – August 10, 1961) was an American author from South Carolina. In 1929 she won the Pulitzer Prize for Novel/Literature, for her novel \"Scarlet Sister Mary.\" She wrote several novels about the plantation South, especially the Gullah people of the Low Country. She was one of the few white authors who wrote about the African-American experience. Julia Mood was born in Laurens County, South Carolina. Her father was a physician, and she was the third of his four children. Her mother died soon after her birth, and her father later married Janie Brogdon.»\n",
- "[2] «E. L. Doctorow | E. L. Doctorow Edgar Lawrence Doctorow (January 6, 1931 – July 21, 2015) was an American novelist, editor, and professor, best known internationally for his works of historical fiction. He has been described as one of the most important American novelists of the 20th century. He wrote twelve novels, three volumes of short fiction and a stage drama. They included the award-winning novels \"Ragtime\" (1975), \"Billy Bathgate\" (1989), and \"The March\" (2005). These, like many of his other works, placed fictional characters in recognizable historical contexts, with known historical figures, and often used different narrative styles. His stories were recognized»\n",
- "[3] «Julia Peterkin | Janie was the mother of Henry Ashleigh Mood, Julia's half-brother and her father's fourth child. He became a doctor. In 1896, at age 16, Julia Mood graduated from Converse College in Spartanburg, South Carolina; she earned her master's degree there a year later. She taught at the public school in Fort Motte, South Carolina, for a few years before she married William George Peterkin in 1903. He was a planter who owned Lang Syne, a cotton plantation near Fort Motte. Julia began writing short stories, inspired by the everyday life and management of the plantation. She was described as audacious»\n",
- "[4] «Julia Peterkin | such conventions. Hurston wrote that she had met Peterkin and would begin a correspondence, but no letters between them have ever been found. Julia Peterkin Julia Peterkin (October 31, 1880 – August 10, 1961) was an American author from South Carolina. In 1929 she won the Pulitzer Prize for Novel/Literature, for her novel \"Scarlet Sister Mary.\" She wrote several novels about the plantation South, especially the Gullah people of the Low Country. She was one of the few white authors who wrote about the African-American experience. Julia Mood was born in Laurens County, South Carolina. Her father was a physician,»\n",
- "[5] «E. L. Doctorow | Fe-Bornstein . He died of lung cancer on July 21, 2015, aged 84, in Manhattan. He is interred in Woodlawn Cemetery in the Bronx. E. L. Doctorow Edgar Lawrence Doctorow (January 6, 1931 – July 21, 2015) was an American novelist, editor, and professor, best known internationally for his works of historical fiction. He has been described as one of the most important American novelists of the 20th century. He wrote twelve novels, three volumes of short fiction and a stage drama. They included the award-winning novels \"Ragtime\" (1975), \"Billy Bathgate\" (1989), and \"The March\" (2005). These, like many of»\n",
- "\n",
- "Question: Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?\n",
- "\n",
- "Rationale: Let's think step by step. We now know that\u001b[32m Julia Peterkin was an American author from South Carolina and E. L. Doctorow was an American novelist, editor, and professor.\n",
- "\n",
- "Search Query: Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?\u001b[0m\u001b[31m \t (and 9 other completions)\u001b[0m\n",
- "\n",
+ "Question: What year was the party of the winner of the 1971 San Francisco mayoral election founded?\n",
+ "Rationale: Let's think step by step. To answer this question, we first need to find out\u001b[32m the name of the party of the winner of the 1971 San Francisco mayoral election.\n",
+ "Search Query: \"Party of the winner of the 1971 San Francisco mayoral election\"\u001b[0m\u001b[31m \t (and 9 other completions)\u001b[0m\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
- "Answer questions with short factoid answers.\n",
- "\n",
- "---\n",
"\n",
- "Question: Who produced the album that included a re-recording of \"Lithium\"?\n",
- "Final Answer: Butch Vig\n",
- "\n",
- "Question: What city was the victim of Joseph Druces working in?\n",
- "Final Answer: Boston, Massachusetts\n",
- "\n",
- "Question: In what year was the star of To Hell and Back born?\n",
- "Final Answer: 1925\n",
+ "Write a search query that will help answer a complex question.\n",
"\n",
"---\n",
"\n",
@@ -1654,64 +1621,60 @@
"\n",
"Question: ${the question to be answered}\n",
"\n",
- "Rationale: Let's think step by step. ${a step-by-step deduction that identifies the correct response, which will be provided below}\n",
+ "Rationale: Let's think step by step. We now know that ${information from the context that may provide useful clues for the question}\n",
"\n",
- "Final Answer: ${a short factoid answer, often between 1 and 5 words}\n",
+ "Search Query: ${a simple question for seeking the missing or required information}\n",
"\n",
"---\n",
"\n",
"Context:\n",
"[1] «Gary Zukav | Gary Zukav Gary Zukav (born October 17, 1942) is an American spiritual teacher and the author of four consecutive New York Times Best Sellers. Beginning in 1998, he appeared more than 30 times on \"The Oprah Winfrey Show\" to discuss transformation in human consciousness concepts presented in his book \"The Seat of the Soul\". His first book, \"The Dancing Wu Li Masters\" (1979), won a U.S. National Book Award. Gary Zukav was born in Port Arthur, Texas, and spent his early childhood in San Antonio and Houston. His family moved to Pittsburg, Kansas, while he was in fourth grade. In»\n",
"[2] «Gary Zukav | experience that led to his first book, \"Dancing Wu Li Masters\", written with extensive help from Jack Sarfatti and other physicists he met through Sarfatti as described in David Kaiser's book \"How the Hippies Saved Physics\". He later described this book as his \"first gift to Life\". In 1987 he moved to Mount Shasta, California, where he lived in a cabin as a self-described \"secular monk\" and spent extensive time in the surrounding wilderness. In 1993 he met Linda Francis. They co-founded the Seat of the Soul Institute in 1998 and moved to Ashland, Oregon, in 2000. Christopher Lehmann-Haupt reviewed»\n",
- "[3] «The Dancing Wu Li Masters | The Dancing Wu Li Masters The Dancing Wu Li Masters is a 1979 book by Gary Zukav, a popular science work exploring modern physics, and quantum phenomena in particular. It was awarded a 1980 U.S. National Book Award in category of Science. Although it explores empirical topics in modern physics research, \"The Dancing Wu Li Masters\" gained attention for leveraging metaphors taken from eastern spiritual movements, in particular the Huayen school of Buddhism with the monk Fazang's treatise on The Golden Lion, to explain quantum phenomena and has been regarded by some reviewers as a New Age work, although the»\n",
- "[4] «The Dancing Wu Li Masters | a 1976 physics conference of eastern and western scientists at Esalen Institute, California; and he used the occasion as material for his book. At the conference, it was said that the Chinese term for physics is 'Wu Li', or \"patterns of organic energy.\" Zukav, among others, conceptualized 'physics' as the dance of the Wu Li Masters – teachers of physical essence. Zukav explains the concept further: The Dancing Wu Li Masters The Dancing Wu Li Masters is a 1979 book by Gary Zukav, a popular science work exploring modern physics, and quantum phenomena in particular. It was awarded a 1980»\n",
"\n",
"Question: Which award did the first book of Gary Zukav receive?\n",
"\n",
- "Rationale: Let's think step by step. The first book is \"The Dancing Wu Li Masters\" and it received a U.S. National Book Award.\n",
+ "Rationale: Let's think step by step. We now know that Gary Zukav's first book is \"The Dancing Wu Li Masters\" and it was written with the help of Jack Sarfatti and other physicists.\n",
"\n",
- "Final Answer: U.S. National Book Award\n",
+ "Search Query: What award did \"The Dancing Wu Li Masters\" receive?\n",
"\n",
"---\n",
"\n",
"Context:\n",
"[1] «John du Pont | John du Pont John Eleuthère du Pont (November 22, 1938 – December 9, 2010) was an American philanthropist, heir to the Du Pont family fortune, and a convicted murderer. He had been a published ornithologist, philatelist, conchologist, sports enthusiast, and self-styled wrestling coach. He died in prison while serving a sentence of 30 years for the murder of Dave Schultz. In 1972, du Pont founded and directed the Delaware Museum of Natural History and contributed to Villanova University and other institutions. In the 1980s, he established a wrestling facility at his Foxcatcher Farm after becoming interested in the sport and»\n",
"[2] «Du Pont family | with political and business ventures, as well as philanthropic efforts on behalf of various organizations such as The Jockey Club, the Philadelphia Orchestra, the Delaware Art Museum, and the Grand Opera House. Two family members were the subjects of well-publicized criminal cases. Heir John Eleuthère du Pont was convicted of murdering wrestling coach Dave Schultz in 1996 and heir Robert H. Richards IV was convicted of raping his 3-year-old daughter in 2009. The family is depicted in the 2014 biographical film \"Foxcatcher\" with Steve Carell playing John Eleuthère du Pont and Vanessa Redgrave playing Jean du Pont, the wife of»\n",
- "[3] «John du Pont | Foxcatcher team. Du Pont became a sponsor in wrestling, swimming, track, and the modern pentathlon. He was also involved in promoting a subset of the modern pentathlon (run, swim, shoot) as a separate event. He took up athletics and became a competitive wrestler in his 50s. His only prior wrestling experience was as a freshman in high school. He began competing again at the age of 55 in the 1992 Veteran's World Championships in Cali, Colombia; following that in 1993 in Toronto, Ontario; in 1994 in Rome, Italy; and in 1995 in Sofia, Bulgaria. In August 1988, a problem-plagued wrestling»\n",
- "[4] «John du Pont | stamps while in prison, he was not allowed to bring them there. After his mother's death, du Pont developed the Liseter Hall Farm in Newtown Square as a high-quality wrestling facility for amateur wrestlers. He called the private group \"Team Foxcatcher,\" after his father's noted racing stable. Du Pont established an Olympic swimming and wrestling training center and sponsored competitive events at the estate. He also allowed some people, such as Olympic champion wrestlers Mark Schultz and later his older brother Dave Schultz and his wife, to live in houses on the grounds for years. Dave Schultz also coached the»\n",
"\n",
"Question: The heir to the Du Pont family fortune sponsored what wrestling team?\n",
"\n",
- "Rationale: Let's think step by step. The heir to the Du Pont family fortune was John du Pont. He was convicted of murdering wrestling coach Dave Schultz in 1996. He was also involved in promoting a subset of the modern pentathlon (run, swim, shoot) as a separate event. He took up athletics and became a competitive wrestler in his 50s. He sponsored competitive events at the estate.\n",
+ "Rationale: Let's think step by step. We now know that John du Pont was an American philanthropist and heir to the Du Pont family fortune. He was also a published ornithologist, philatelist, conchologist, sports enthusiast, and self-styled wrestling coach.\n",
"\n",
- "Final Answer: Team Foxcatcher\n",
+ "Search Query: What wrestling team did John du Pont sponsor?\n",
"\n",
"---\n",
"\n",
"Context:\n",
"[1] «Peter Outerbridge | Peter Outerbridge Peter Outerbridge (born June 30, 1966) is a Canadian actor, best known for his role as Ari Tasarov in the CW action series \"Nikita\", Dr. David Sandström in the TMN series \"ReGenesis\", Henrik \"Hank\" Johanssen in \"Orphan Black\", Bob Corbett in \"Bomb Girls\", William Easton in \"Saw VI\" and George Brown in the television film \"\". He also played the lead role of Detective William Murdoch in a three-episode mini-series, \"The Murdoch Mysteries\", in its initial run on Canadian television, with two episodes shown in 2004 and a third in 2005. Outerbridge was born and raised in Toronto,»\n",
"[2] «Except the Dying | 2008. On March 3, 2015, Acorn Media announced a re-release for all three movies, set for May 26, 2015. Except the Dying Except the Dying is a 2004 made-for-TV film starring Peter Outerbridge, Colm Meaney, Keeley Hawes, William B. Davis and Flora Montgomery. It was adapted by Janet MacLean from the novel of the same name by Maureen Jennings. In this film, William Murdoch is introduced, as a man of strong principles, who uses his unique abilities to solve crimes, sometimes using advanced science for his time. On the street of Toronto, in the 1890s, the naked body of a»\n",
- "[3] «Saw VI | Saw VI Saw VI is a 2009 American horror film directed by Kevin Greutert from a screenplay written by Patrick Melton and Marcus Dunstan. It is the sixth installment in the \"Saw\" franchise and stars Tobin Bell, Costas Mandylor, Betsy Russell, Mark Rolston, Peter Outerbridge, and Shawnee Smith. It was produced by Mark Burg and Oren Koules of Twisted Pictures and distributed by Lionsgate. \"Saw VI\" continues the franchise's focus on the posthumous effects of the Jigsaw Killer and the progression of his successor, Mark Hoffman. In this film, Hoffman sets a series of traps for an insurance executive, William»\n",
"\n",
"Question: Who was the director of the 2009 movie featuring Peter Outerbridge as William Easton?\n",
"\n",
- "Rationale: Let's think step by step. We know that Peter Outerbridge was in the movie \"Saw VI\" and that he played the character William Easton. We also know that the movie was released in 2009. From this, we can deduce that the director of the movie was Kevin Greutert.\n",
+ "Rationale: Let's think step by step. We now know that Peter Outerbridge played William Easton in the 2009 movie.\n",
"\n",
- "Final Answer: Kevin Greutert\n",
+ "Search Query: Who was the director of the 2009 movie featuring Peter Outerbridge as William Easton?\n",
"\n",
"---\n",
"\n",
"Context:\n",
- "[1] «Julia Peterkin was an American author from South Carolina and E. L. Doctorow was an American novelist, editor, and professor.»\n",
- "[2] «Julia Peterkin | Julia Peterkin Julia Peterkin (October 31, 1880 – August 10, 1961) was an American author from South Carolina. In 1929 she won the Pulitzer Prize for Novel/Literature, for her novel \"Scarlet Sister Mary.\" She wrote several novels about the plantation South, especially the Gullah people of the Low Country. She was one of the few white authors who wrote about the African-American experience. Julia Mood was born in Laurens County, South Carolina. Her father was a physician, and she was the third of his four children. Her mother died soon after her birth, and her father later married Janie Brogdon.»\n",
- "[3] «E. L. Doctorow | E. L. Doctorow Edgar Lawrence Doctorow (January 6, 1931 – July 21, 2015) was an American novelist, editor, and professor, best known internationally for his works of historical fiction. He has been described as one of the most important American novelists of the 20th century. He wrote twelve novels, three volumes of short fiction and a stage drama. They included the award-winning novels \"Ragtime\" (1975), \"Billy Bathgate\" (1989), and \"The March\" (2005). These, like many of his other works, placed fictional characters in recognizable historical contexts, with known historical figures, and often used different narrative styles. His stories were recognized»\n",
- "[4] «Physician | scope of practice for a nurse practitioner in the United States is defined by regulatory boards of nursing, as opposed to boards of medicine that regulate physicians. Physician A physician, medical practitioner, medical doctor, or simply doctor is a professional who practises medicine, which is concerned with promoting, maintaining, or restoring health through the study, diagnosis, and treatment of disease, injury, and other physical and mental impairments. Physicians may focus their practice on certain disease categories, types of patients and methods of treatment—known as specialities—or they may assume responsibility for the provision of continuing and comprehensive medical care to individuals,»\n",
- "[5] «E. L. Doctorow | Utah, the University of California, Irvine, and Princeton University. He was the Loretta and Lewis Glucksman Professor of English and American Letters at New York University. In 2001 he donated his papers to the Fales Library of New York University. The library's director, Marvin Taylor, said Doctorow was \"one of the most important American novelists of the 20th century\". In 1954, Doctorow married fellow Columbia University student Helen Esther Setzer while serving in the U.S. Army in West Germany. The couple had three children: television writer and producer Richard Doctorow, singer-songwriter and record producer Caroline Doctorow Gatewood and Jenny Doctorow»\n",
- "[6] «Julia Peterkin | to Alfred Knopf, who published her first book, \"Green Thursday,\" in 1924. In addition to a number of subsequent novels, her short stories were published in magazines and newspaper throughout her career. Peterkin was among the few white authors to specialize in the African-American experience. She won a Pulitzer Prize in 1929 for her novel, \"Scarlet Sister Mary.\" Dr. Richard S. Burton, the chairperson of Pulitzer's fiction-literature jury, recommended that the first prize go to the novel \"Victim and Victor\" by John Rathbone Oliver. The School of Journalism chose Peterkin's book. Burton resigned from the jury. The book aroused some»\n",
+ "[1] «1971 San Francisco mayoral election | 1971 San Francisco mayoral election The San Francisco mayoral election of 1971 was held on November 2, 1971, with incumbent Joseph Alioto being re-elected with 38.6 percent of the vote, from among 11 candidates, there being no provision for a runoff. The three major contenders—Joe Alioto; Harold Dobbs, former member of the Board of Supervisors; and Dianne Feinstein, president of the Board of Supervisors—tried to outdo one another as advocates for law and order. Alioto challenged Dobbs to commit to keeping Police Chief Donald Scott and accused Feinstein of wanting to create a civilian review board to investigate police conduct.»\n",
+ "[2] «1971 San Francisco mayoral election | the November 2 election was the re-election of Mayor Joseph Alioto by a 3–2 margin over his nearest challenger. Alioto quoted Jackie Gleason—\"How sweet it is!\"—at a victory celebration and referred to his federal indictment as an attempt by the \"national administration\" to \"knock out a mayor\". Among other races on the ballot was Richard Hongisto's surprise victory for sheriff over incumbent Matthew Carberry. On the ballot was a referendum limiting new-building heights to six stories, which was defeated. A referendum to make the school board an elective one, it being the only appointive board in the state, passed. The»\n",
+ "[3] «1971 San Francisco mayoral election | school-board measure was regarded as a reaction to busing initiated by the school board under court order. Another measure approved was to keep cable car service at no less than current levels. 1971 San Francisco mayoral election The San Francisco mayoral election of 1971 was held on November 2, 1971, with incumbent Joseph Alioto being re-elected with 38.6 percent of the vote, from among 11 candidates, there being no provision for a runoff. The three major contenders—Joe Alioto; Harold Dobbs, former member of the Board of Supervisors; and Dianne Feinstein, president of the Board of Supervisors—tried to outdo one another»\n",
+ "[4] «1971 San Francisco mayoral election | achieve ethnic balance in elementary schools. Former newspaper editor Scott Newhall proposed staging a world ecology fair and replacing California's water plan by importing icebergs, and imputed little difference between the top three contenders, whom he lumped together as \"Joe Dobbstein\". Alioto was a slight favorite to win in spite of being scheduled to go on trial the following January on charges of conspiracy, mail fraud, and using the mail to promote bribery of a public official. As of November, Dobbs, the only Republican in the race, had passed Feinstein in polls and was gaining on Alioto. The result of»\n",
+ "[5] «2003 San Francisco mayoral election | against leftist/progressive Gonzalez, a member of the Green Party. It was the first mayoral election in San Francisco that a Green Party candidate took a noticeable amount of the vote. The election was close, with Gonzalez leading in the polls and winning the popular vote among ballots cast on election day, while Newsom had a larger lead on absentee ballots. 2003 San Francisco mayoral election The 2003 San Francisco mayoral election occurred on November 4, 2003. The incumbent, Willie Brown, was termed out of office and could not seek a third term. The general election included three top candidates including»\n",
"\n",
- "Question: Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?\n",
+ "Question: What year was the party of the winner of the 1971 San Francisco mayoral election founded?\n",
"\n",
- "Rationale: Let's think step by step.\u001b[32m We are given information about the professions of E. L. Doctorow and Julia Peterkin. E. L. Doctorow was an American novelist, editor, and professor. Julia Peterkin was an American author. Based on this, we can deduce that the scope of profession for E. L. Doctorow is broader than that of Julia Peterkin.\u001b[0m\u001b[31m \t (and 19 other completions)\u001b[0m\n",
+ "Rationale: Let's think step by step. We now know that\u001b[32m the winner of the 1971 San Francisco mayoral election was Joseph Alioto and he was a member of the Democratic Party.\n",
+ "\n",
+ "Search Query: When was the Democratic Party founded?\u001b[0m\u001b[31m \t (and 9 other completions)\u001b[0m\n",
"\n",
"\n",
"\n",
@@ -1789,18 +1752,20 @@
"---\n",
"\n",
"Context:\n",
- "[1] «Julia Peterkin was an American author from South Carolina and E. L. Doctorow was an American novelist, editor, and professor.»\n",
- "[2] «Julia Peterkin | Julia Peterkin Julia Peterkin (October 31, 1880 – August 10, 1961) was an American author from South Carolina. In 1929 she won the Pulitzer Prize for Novel/Literature, for her novel \"Scarlet Sister Mary.\" She wrote several novels about the plantation South, especially the Gullah people of the Low Country. She was one of the few white authors who wrote about the African-American experience. Julia Mood was born in Laurens County, South Carolina. Her father was a physician, and she was the third of his four children. Her mother died soon after her birth, and her father later married Janie Brogdon.»\n",
- "[3] «E. L. Doctorow | E. L. Doctorow Edgar Lawrence Doctorow (January 6, 1931 – July 21, 2015) was an American novelist, editor, and professor, best known internationally for his works of historical fiction. He has been described as one of the most important American novelists of the 20th century. He wrote twelve novels, three volumes of short fiction and a stage drama. They included the award-winning novels \"Ragtime\" (1975), \"Billy Bathgate\" (1989), and \"The March\" (2005). These, like many of his other works, placed fictional characters in recognizable historical contexts, with known historical figures, and often used different narrative styles. His stories were recognized»\n",
- "[4] «Physician | scope of practice for a nurse practitioner in the United States is defined by regulatory boards of nursing, as opposed to boards of medicine that regulate physicians. Physician A physician, medical practitioner, medical doctor, or simply doctor is a professional who practises medicine, which is concerned with promoting, maintaining, or restoring health through the study, diagnosis, and treatment of disease, injury, and other physical and mental impairments. Physicians may focus their practice on certain disease categories, types of patients and methods of treatment—known as specialities—or they may assume responsibility for the provision of continuing and comprehensive medical care to individuals,»\n",
- "[5] «E. L. Doctorow | Utah, the University of California, Irvine, and Princeton University. He was the Loretta and Lewis Glucksman Professor of English and American Letters at New York University. In 2001 he donated his papers to the Fales Library of New York University. The library's director, Marvin Taylor, said Doctorow was \"one of the most important American novelists of the 20th century\". In 1954, Doctorow married fellow Columbia University student Helen Esther Setzer while serving in the U.S. Army in West Germany. The couple had three children: television writer and producer Richard Doctorow, singer-songwriter and record producer Caroline Doctorow Gatewood and Jenny Doctorow»\n",
- "[6] «Julia Peterkin | to Alfred Knopf, who published her first book, \"Green Thursday,\" in 1924. In addition to a number of subsequent novels, her short stories were published in magazines and newspaper throughout her career. Peterkin was among the few white authors to specialize in the African-American experience. She won a Pulitzer Prize in 1929 for her novel, \"Scarlet Sister Mary.\" Dr. Richard S. Burton, the chairperson of Pulitzer's fiction-literature jury, recommended that the first prize go to the novel \"Victim and Victor\" by John Rathbone Oliver. The School of Journalism chose Peterkin's book. Burton resigned from the jury. The book aroused some»\n",
+ "[1] «the winner of the 1971 San Francisco mayoral election was Joseph Alioto and he was a member of the Democratic Party.»\n",
+ "[2] «Democratic Party (UK, 1998) | Democratic Party (UK, 1998) The Democratic Party was a political party in the United Kingdom, founded in November 1998 by Malvern businessman Geoff Southall, who also funded the party. It was described as \"right wing\" or \"hard right\" in news reports, but claimed to occupy the centre-ground of British politics. It aimed to reduce Britain's involvement with the European Union, opposed the adoption of the Euro, called for direct democracy, and argued for limits on immigration. The slogan was \"The will of the people NOT the party\". In 1999 it had a few hundred members, including previous supporters of James»\n",
+ "[3] «Democratic Party (Brazil, 1925-1934) | Democratic Party (Brazil, 1925-1934) The Democratic Party was a Brazilian political party, but state character, founded by dissidents of the Paulista Republican Party (PRP) on February 24, 1925, during the Old Republic and was its manifesto signed by 599 signatories. It was a representative organization of the traditional middle class, linked to coffee sectors, but especially the urban. In May 1926, the Youth Party (Youth Wing of dissenters, who had organized this caption for the federal elections of 1925), also dissident perrepismo, joined the Democratic Party. His most important names were Waldemar Ferreira, José Adriano Marrey Junior, Francisco Morato, Paulo»\n",
+ "[4] «Democratic Party (Turkey, 1970) | Democratic Party (Turkey, 1970) Democratic Party (Turkish: \"Demokratik Parti\", DP for short) was a former political party in Turkey founded on 18 December 1970 . It should not be confused with the historical Democrat Party or the current Democrat Party in Turkey. The historical Democrat Party which was the ruling party between 1950-1960 was closed by the military rule. In 1961 and 1965 elections most of ex Democratic Party adherents supported Justice Party, a party founded in 1961. After 1965 elections Justice Party came to power. But following 1969 elections in which Justice Party was again successful, tension between the»\n",
+ "[5] «Democratic Party (Japan, 1947) | Democratic Party (Japan, 1947) Democratic Party (民主党, Minshutō) was a centrist political party in Japan. It was founded in spring 1947 by merging the Progressive Party (Shinpo-tō) of Inukai Takeru with a faction of Liberal Party led by Hitoshi Ashida and obtained 124 seats in 1947 elections. The party had held seven seats in Tetsu Katayama's government in 1947-1948. For some months in 1948, party's leader Ashida was Prime minister. In March 1948, part of DP members led by Kijūrō Shidehara joined the Liberal Party to form the Democratic Liberal Party. In 1949 elections, the DP got 69 seats. The»\n",
+ "[6] «Democratic Party (United States) | Democratic Party (United States) The Democratic Party is one of the two major contemporary political parties in the United States, along with the Republican Party. Tracing its heritage back to Thomas Jefferson and James Madison's Democratic-Republican Party, the modern-day Democratic Party was founded around 1828 by supporters of Andrew Jackson, making it the world's oldest active political party. The Democrats' dominant worldview was once social conservatism and economic liberalism while populism was its leading characteristic in the rural South. In 1912, Theodore Roosevelt ran as a third-party candidate in the Progressive (\"Bull Moose\") Party, beginning a switch of political platforms»\n",
"\n",
- "Question: Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?\n",
+ "Question: What year was the party of the winner of the 1971 San Francisco mayoral election founded?\n",
"\n",
- "Rationale: Let's think step by step. We are given information about the professions of E. L. Doctorow and Julia Peterkin. E. L. Doctorow was an American novelist, editor, and professor. Julia Peterkin was an American author. Based on this, we can deduce that the scope of profession for E. L. Doctorow is broader than that of Julia Peterkin.\n",
+ "Rationale: Let's think step by step.\u001b[32m We know that Joseph Alioto was the winner of the 1971 San Francisco mayoral election and that he was a member of the Democratic Party. We also know that the Democratic Party was founded in 1828. From this, we can deduce that the party was founded in 1828.\n",
"\n",
- "Final Answer:\u001b[32m E. L. Doctorow\u001b[0m\n",
+ "Final Answer:\n",
+ "\n",
+ "1828\u001b[0m\u001b[31m \t (and 19 other completions)\u001b[0m\n",
"\n",
"\n",
"\n"
@@ -1809,7 +1774,7 @@
{
"data": {
"text/plain": [
- "('E. L. Doctorow', None)"
+ "('1828', None)"
]
},
"execution_count": 24,
@@ -1818,7 +1783,7 @@
}
],
"source": [
- "multihop_QA_v2(dev[0].question), lm.inspect_history(n=3)"
+ "multihop_QA_v2(dev[3].question), lm.inspect_history(n=3)"
]
},
{
@@ -1830,7 +1795,7 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "100%|██████████| 13/13 [00:03<00:00, 4.09it/s]"
+ "100%|██████████| 13/13 [00:01<00:00, 8.38it/s]"
]
},
{
@@ -1851,14 +1816,14 @@
"data": {
"text/html": [
"<style type=\"text/css\">\n",
- "#T_7b33b_ th {\n",
+ "#T_b8b4e_ th {\n",
" text-align: left;\n",
"}\n",
- "#T_7b33b_ td {\n",
+ "#T_b8b4e_ td {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
- "<table id=\"T_7b33b_\">\n",
+ "<table id=\"T_b8b4e_\">\n",
" <thead>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
@@ -1870,101 +1835,101 @@
" </thead>\n",
" <tbody>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
- " <td id=\"T_7b33b_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
- " <td id=\"T_7b33b_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
- " <td id=\"T_7b33b_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
- " <td id=\"T_7b33b_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
+ " <td id=\"T_b8b4e_row0_col0\" class=\"data row0 col0\" >Who has a broader scope of profession: E. L. Doctorow or Julia Peterkin?</td>\n",
+ " <td id=\"T_b8b4e_row0_col1\" class=\"data row0 col1\" >['E. L. Doctorow', 'E.L. Doctorow', 'Doctorow']</td>\n",
+ " <td id=\"T_b8b4e_row0_col2\" class=\"data row0 col2\" >E. L. Doctorow</td>\n",
+ " <td id=\"T_b8b4e_row0_col3\" class=\"data row0 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
- " <td id=\"T_7b33b_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
- " <td id=\"T_7b33b_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
- " <td id=\"T_7b33b_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
- " <td id=\"T_7b33b_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
+ " <td id=\"T_b8b4e_row1_col0\" class=\"data row1 col0\" >What documentary about the Gilgo Beach Killer debuted on A&E?</td>\n",
+ " <td id=\"T_b8b4e_row1_col1\" class=\"data row1 col1\" >['The Killing Season']</td>\n",
+ " <td id=\"T_b8b4e_row1_col2\" class=\"data row1 col2\" >The Killing Season</td>\n",
+ " <td id=\"T_b8b4e_row1_col3\" class=\"data row1 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
- " <td id=\"T_7b33b_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
- " <td id=\"T_7b33b_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
- " <td id=\"T_7b33b_row2_col2\" class=\"data row2 col2\" >Gainesville, Florida</td>\n",
- " <td id=\"T_7b33b_row2_col3\" class=\"data row2 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
+ " <td id=\"T_b8b4e_row2_col0\" class=\"data row2 col0\" >Right Back At It Again contains lyrics co-written by the singer born in what city?</td>\n",
+ " <td id=\"T_b8b4e_row2_col1\" class=\"data row2 col1\" >['Gainesville, Florida', 'Gainesville']</td>\n",
+ " <td id=\"T_b8b4e_row2_col2\" class=\"data row2 col2\" >Gainesville, Florida</td>\n",
+ " <td id=\"T_b8b4e_row2_col3\" class=\"data row2 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
- " <td id=\"T_7b33b_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
- " <td id=\"T_7b33b_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
- " <td id=\"T_7b33b_row3_col2\" class=\"data row3 col2\" >1828</td>\n",
- " <td id=\"T_7b33b_row3_col3\" class=\"data row3 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
+ " <td id=\"T_b8b4e_row3_col0\" class=\"data row3 col0\" >What year was the party of the winner of the 1971 San Francisco mayoral election founded?</td>\n",
+ " <td id=\"T_b8b4e_row3_col1\" class=\"data row3 col1\" >['1828']</td>\n",
+ " <td id=\"T_b8b4e_row3_col2\" class=\"data row3 col2\" >1828</td>\n",
+ " <td id=\"T_b8b4e_row3_col3\" class=\"data row3 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
- " <td id=\"T_7b33b_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
- " <td id=\"T_7b33b_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
- " <td id=\"T_7b33b_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
- " <td id=\"T_7b33b_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
+ " <td id=\"T_b8b4e_row4_col0\" class=\"data row4 col0\" >Which author is English: John Braine or Studs Terkel?</td>\n",
+ " <td id=\"T_b8b4e_row4_col1\" class=\"data row4 col1\" >['John Braine']</td>\n",
+ " <td id=\"T_b8b4e_row4_col2\" class=\"data row4 col2\" >John Braine</td>\n",
+ " <td id=\"T_b8b4e_row4_col3\" class=\"data row4 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
- " <td id=\"T_7b33b_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
- " <td id=\"T_7b33b_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
- " <td id=\"T_7b33b_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
- " <td id=\"T_7b33b_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
+ " <td id=\"T_b8b4e_row5_col0\" class=\"data row5 col0\" >Anthony Dirrell is the brother of which super middleweight title holder?</td>\n",
+ " <td id=\"T_b8b4e_row5_col1\" class=\"data row5 col1\" >['Andre Dirrell']</td>\n",
+ " <td id=\"T_b8b4e_row5_col2\" class=\"data row5 col2\" >Andre Dirrell</td>\n",
+ " <td id=\"T_b8b4e_row5_col3\" class=\"data row5 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
- " <td id=\"T_7b33b_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
- " <td id=\"T_7b33b_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
- " <td id=\"T_7b33b_row6_col2\" class=\"data row6 col2\" >Cheshire</td>\n",
- " <td id=\"T_7b33b_row6_col3\" class=\"data row6 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
+ " <td id=\"T_b8b4e_row6_col0\" class=\"data row6 col0\" >In which city is the sports nutrition business established by Oliver Cookson based ?</td>\n",
+ " <td id=\"T_b8b4e_row6_col1\" class=\"data row6 col1\" >['Cheshire', 'Cheshire, UK']</td>\n",
+ " <td id=\"T_b8b4e_row6_col2\" class=\"data row6 col2\" >Cheshire</td>\n",
+ " <td id=\"T_b8b4e_row6_col3\" class=\"data row6 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
- " <td id=\"T_7b33b_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
- " <td id=\"T_7b33b_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
- " <td id=\"T_7b33b_row7_col2\" class=\"data row7 col2\" >1958</td>\n",
- " <td id=\"T_7b33b_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
+ " <th id=\"T_b8b4e_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
+ " <td id=\"T_b8b4e_row7_col0\" class=\"data row7 col0\" >Find the birth date of the actor who played roles in First Wives Club and Searching for the Elephant.</td>\n",
+ " <td id=\"T_b8b4e_row7_col1\" class=\"data row7 col1\" >['February 13, 1980']</td>\n",
+ " <td id=\"T_b8b4e_row7_col2\" class=\"data row7 col2\" >1958</td>\n",
+ " <td id=\"T_b8b4e_row7_col3\" class=\"data row7 col3\" >❌</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
- " <td id=\"T_7b33b_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
- " <td id=\"T_7b33b_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
- " <td id=\"T_7b33b_row8_col2\" class=\"data row8 col2\" >Castletown River</td>\n",
- " <td id=\"T_7b33b_row8_col3\" class=\"data row8 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
+ " <td id=\"T_b8b4e_row8_col0\" class=\"data row8 col0\" >Kyle Moran was born in the town on what river?</td>\n",
+ " <td id=\"T_b8b4e_row8_col1\" class=\"data row8 col1\" >['Castletown', 'Castletown River']</td>\n",
+ " <td id=\"T_b8b4e_row8_col2\" class=\"data row8 col2\" >Castletown River</td>\n",
+ " <td id=\"T_b8b4e_row8_col3\" class=\"data row8 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
- " <td id=\"T_7b33b_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
- " <td id=\"T_7b33b_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
- " <td id=\"T_7b33b_row9_col2\" class=\"data row9 col2\" >Aeronautics</td>\n",
- " <td id=\"T_7b33b_row9_col3\" class=\"data row9 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
+ " <td id=\"T_b8b4e_row9_col0\" class=\"data row9 col0\" >What is the name of one branch of Robert D. Braun's speciality?</td>\n",
+ " <td id=\"T_b8b4e_row9_col1\" class=\"data row9 col1\" >['aeronautical engineering', 'astronautical engineering', 'aeronautics', 'astronautics']</td>\n",
+ " <td id=\"T_b8b4e_row9_col2\" class=\"data row9 col2\" >Aeronautics</td>\n",
+ " <td id=\"T_b8b4e_row9_col3\" class=\"data row9 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
- " <td id=\"T_7b33b_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
- " <td id=\"T_7b33b_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
- " <td id=\"T_7b33b_row10_col2\" class=\"data row10 col2\" >Surrey</td>\n",
- " <td id=\"T_7b33b_row10_col3\" class=\"data row10 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
+ " <td id=\"T_b8b4e_row10_col0\" class=\"data row10 col0\" >Where was the actress who played the niece in the Priest film born?</td>\n",
+ " <td id=\"T_b8b4e_row10_col1\" class=\"data row10 col1\" >['Surrey', 'Guildford, Surrey']</td>\n",
+ " <td id=\"T_b8b4e_row10_col2\" class=\"data row10 col2\" >Surrey</td>\n",
+ " <td id=\"T_b8b4e_row10_col3\" class=\"data row10 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
- " <td id=\"T_7b33b_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
- " <td id=\"T_7b33b_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
- " <td id=\"T_7b33b_row11_col2\" class=\"data row11 col2\" >Portrait of a Marriage</td>\n",
- " <td id=\"T_7b33b_row11_col3\" class=\"data row11 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
+ " <td id=\"T_b8b4e_row11_col0\" class=\"data row11 col0\" >Name the movie in which the daughter of Noel Harrison plays Violet Trefusis.</td>\n",
+ " <td id=\"T_b8b4e_row11_col1\" class=\"data row11 col1\" >['Portrait of a Marriage']</td>\n",
+ " <td id=\"T_b8b4e_row11_col2\" class=\"data row11 col2\" >Portrait of a Marriage</td>\n",
+ " <td id=\"T_b8b4e_row11_col3\" class=\"data row11 col3\" >✔️</td>\n",
" </tr>\n",
" <tr>\n",
- " <th id=\"T_7b33b_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
- " <td id=\"T_7b33b_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
- " <td id=\"T_7b33b_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
- " <td id=\"T_7b33b_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
- " <td id=\"T_7b33b_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
+ " <th id=\"T_b8b4e_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
+ " <td id=\"T_b8b4e_row12_col0\" class=\"data row12 col0\" >What year was the father of the Princes in the Tower born?</td>\n",
+ " <td id=\"T_b8b4e_row12_col1\" class=\"data row12 col1\" >['1442']</td>\n",
+ " <td id=\"T_b8b4e_row12_col2\" class=\"data row12 col2\" >1442</td>\n",
+ " <td id=\"T_b8b4e_row12_col3\" class=\"data row12 col3\" >✔️</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
- "<pandas.io.formats.style.Styler at 0x7f090dbbff10>"
+ "<pandas.io.formats.style.Styler at 0x7fa01d1dbb50>"
]
},
"metadata": {},