Skip to content

armgit / storm

Public Imported from GitHub
0

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

Create eval_prometheus_no_ref.prompt

Yucheng-Jiang3e08ca233ee7
diff --git a/eval/eval_prometheus_no_ref.prompt b/eval/eval_prometheus_no_ref.prompt
new file mode 100644
index 0000000..b008bdc
--- /dev/null
+++ b/eval/eval_prometheus_no_ref.prompt
@@ -0,0 +1,22 @@
+###Task Description:
+An instruction (might include an Input inside it), a response to evaluate, a reference answer that gets a score of 5, and a score rubric representing a evaluation criteria are given.
+1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.
+2. After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.
+3. The output format should look as follows: \"Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)\".
+4. Please do not generate any other opening, closing, and explanations.
+
+###The instruction to evaluate:
+{instruction}
+
+###Response to evaluate:
+{response}
+
+###Score Rubrics:
+[{criteria_description}]
+Score 1: {score1_description}
+Score 2: {score2_description}
+Score 3: {score3_description}
+Score 4: {score4_description}
+Score 5: {score5_description}
+
+###Feedback: