An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.
Python agentic-ragdeep-researchemnlp2024knowledge-curationlarge-language-modelsnaaclnlpreport-generationretrieval-augmented-generation
Create eval_prometheus_no_ref.prompt
Yucheng-Jiang3e08ca233ee7
diff --git a/eval/eval_prometheus_no_ref.prompt b/eval/eval_prometheus_no_ref.promptnew file mode 100644index 0000000..b008bdc--- /dev/null+++ b/eval/eval_prometheus_no_ref.prompt@@ -0,0 +1,22 @@+###Task Description:+An instruction (might include an Input inside it), a response to evaluate, a reference answer that gets a score of 5, and a score rubric representing a evaluation criteria are given.+1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.+2. After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.+3. The output format should look as follows: \"Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)\".+4. Please do not generate any other opening, closing, and explanations.++###The instruction to evaluate:+{instruction}++###Response to evaluate:+{response}++###Score Rubrics:+[{criteria_description}]+Score 1: {score1_description}+Score 2: {score2_description}+Score 3: {score3_description}+Score 4: {score4_description}+Score 5: {score5_description}++###Feedback: