An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.
Python agentic-ragdeep-researchemnlp2024knowledge-curationlarge-language-modelsnaaclnlpreport-generationretrieval-augmented-generation
Fix organization name for prometheus model
Scott Suk743752722bf0
diff --git a/README.md b/README.mdindex d483e72..76fd58a 100644--- a/README.md+++ b/README.md@@ -222,7 +222,7 @@ python eval_article_quality.py --input-path ../FreshWiki/topic_list.csv --gt-dir#### Use the Metric YourselfThe similarity-based metrics (i.e., ROUGE, entity recall, and heading entity recall) are implemented in [eval/metrics.py](eval/metrics.py).-For rubric grading, we use the [prometheus-13b-v1.0](https://huggingface.co/kaist-ai/prometheus-13b-v1.0) introduced in [this paper](https://arxiv.org/abs/2310.08491). [eval/evaluation_prometheus.py](eval/evaluation_prometheus.py) provides the entry point of using the metric.+For rubric grading, we use the [prometheus-13b-v1.0](https://huggingface.co/prometheus-eval/prometheus-13b-v1.0) introduced in [this paper](https://arxiv.org/abs/2310.08491). [eval/evaluation_prometheus.py](eval/evaluation_prometheus.py) provides the entry point of using the metric.</details>diff --git a/eval/eval_article_quality.py b/eval/eval_article_quality.pyindex bf24f49..521f606 100644--- a/eval/eval_article_quality.py+++ b/eval/eval_article_quality.py@@ -176,8 +176,8 @@ if __name__ == "__main__":parser.add_argument('--tokenizer', default="meta-llama/Llama-2-7b-chat-hf")parser.add_argument('--model',- choices=["kaist-ai/prometheus-13b-v1.0", "kaist-ai/prometheus-7b-v1.0"],- default="kaist-ai/prometheus-13b-v1.0",+ choices=["prometheus-eval/prometheus-13b-v1.0", "prometheus-eval/prometheus-7b-v1.0"],+ default="prometheus-eval/prometheus-13b-v1.0",help="Model to use for rubric evaluation.")args = parser.parse_args()diff --git a/eval/evaluation_prometheus.py b/eval/evaluation_prometheus.pyindex 3f20a9c..2569bd9 100644--- a/eval/evaluation_prometheus.py+++ b/eval/evaluation_prometheus.py@@ -171,9 +171,9 @@ if __name__ == "__main__":parser.add_argument('--tokenizer', default="meta-llama/Llama-2-7b-chat-hf")parser.add_argument('--model',- choices=["kaist-ai/prometheus-13b-v1.0", "kaist-ai/prometheus-7b-v1.0"],- default="kaist-ai/prometheus-13b-v1.0",- help="Model to use; options are 'kaist-ai/prometheus-13b-v1.0' or 'kaist-ai/prometheus-7b-v1.0'")+ choices=["prometheus-eval/prometheus-13b-v1.0", "prometheus-eval/prometheus-7b-v1.0"],+ default="prometheus-eval/prometheus-13b-v1.0",+ help="Model to use; options are 'prometheus-eval/prometheus-13b-v1.0' or 'prometheus-eval/prometheus-7b-v1.0'")parser.add_argument('--disable_sample', action='store_true', help='Whether to disable sampling; default is False')parser.add_argument('--temperature', type=float, default=0.01, help='Temperature for generation; default is 0.01')parser.add_argument('--top_p', type=float, default=0.95, help='Top P for generation; default is 0.95')