DSPy: The framework for programming—not prompting—language models
Python
Update longformqa_assertions.ipynb
arnavsinghvi11a166d6022aa3
diff --git a/examples/longformqa/longformqa_assertions.ipynb b/examples/longformqa/longformqa_assertions.ipynbindex f3ae4326a..8d7d2bdb4 100644--- a/examples/longformqa/longformqa_assertions.ipynb+++ b/examples/longformqa/longformqa_assertions.ipynb@@ -569,7 +569,7 @@"source": ["Let's evaluate this with assertions. \n","\n",- "**Note** The pipeline here lies in compiling without assertions to give the teleprompter correct bootstrapped examples by the `answer_correctness` metric and then adding **Assertions** post-compilation. This is represented by passing `LongFormQA()` as the teacher and `LongFormQAWithAssertions()` as the student."+ "**Note** The pipeline here lies in compiling with **Assertions** to give the teleprompter correct bootstrapped examples by the `answer_correctness` metric and then 'teaching' the student with these correct examples. This is represented by passing `LongFormQA()` as the student and `LongFormQAWithAssertions()` as the teacher."]},{@@ -580,10 +580,29 @@"source": ["longformqa = LongFormQA()\n","teleprompter = BootstrapFewShotWithRandomSearch(metric = answer_correctness, max_bootstrapped_demos=2, num_candidate_programs=6)\n",- "cited_longformqa = teleprompter.compile(student=assert_transform_module(LongFormQAWithAssertions().map_named_predictors(Retry), backtrack_handler), teacher = longformqa, trainset=trainset, valset=devset[:100])\n",- "evaluate(cited_longformqa)"- ]- }+ "cited_longformqa_teacher = teleprompter.compile(student=longformqa, teacher = assert_transform_module(LongFormQAWithAssertions().map_named_predictors(Retry), backtrack_handler), trainset=trainset, valset=devset[:100])",+ "evaluate(cited_longformqa_teacher)"+ ]+ },+ {+ "cell_type": "markdown",+ "metadata": {},+ "source": [+ "**Note** This pipeline on the other hand sets both the teacher and student with `LongFormQAWithAssertions()` to ensure the teacher correctly instructs the student with the right bootstrapped examples and the student has the chance to self-correct with **Assertions** for any examples that are still deemed incorrect."+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "metadata": {},+ "outputs": [],+ "source": [+ "longformqa = LongFormQA()\n",+ "teleprompter = BootstrapFewShotWithRandomSearch(metric = answer_correctness, max_bootstrapped_demos=2, num_candidate_programs=6)\n",+ "cited_longformqa_student_teacher = teleprompter.compile(student=assert_transform_module(LongFormQAWithAssertions().map_named_predictors(Retry), backtrack_handler), teacher = assert_transform_module(LongFormQAWithAssertions().map_named_predictors(Retry), backtrack_handler), trainset=trainset, valset=devset[:100])",+ "evaluate(cited_longformqa_student_teacher)"+ ]+ },],"metadata": {"kernelspec": {