Skip to content

armgit / storm

Public Imported from GitHub
0

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

Fix organization name for prometheus model

Scott Suk743752722bf0
diff --git a/README.md b/README.md
index d483e72..76fd58a 100644
--- a/README.md
+++ b/README.md
@@ -222,7 +222,7 @@ python eval_article_quality.py --input-path ../FreshWiki/topic_list.csv --gt-dir
#### Use the Metric Yourself
The similarity-based metrics (i.e., ROUGE, entity recall, and heading entity recall) are implemented in [eval/metrics.py](eval/metrics.py).
-For rubric grading, we use the [prometheus-13b-v1.0](https://huggingface.co/kaist-ai/prometheus-13b-v1.0) introduced in [this paper](https://arxiv.org/abs/2310.08491). [eval/evaluation_prometheus.py](eval/evaluation_prometheus.py) provides the entry point of using the metric.
+For rubric grading, we use the [prometheus-13b-v1.0](https://huggingface.co/prometheus-eval/prometheus-13b-v1.0) introduced in [this paper](https://arxiv.org/abs/2310.08491). [eval/evaluation_prometheus.py](eval/evaluation_prometheus.py) provides the entry point of using the metric.
</details>
diff --git a/eval/eval_article_quality.py b/eval/eval_article_quality.py
index bf24f49..521f606 100644
--- a/eval/eval_article_quality.py
+++ b/eval/eval_article_quality.py
@@ -176,8 +176,8 @@ if __name__ == "__main__":
parser.add_argument('--tokenizer', default="meta-llama/Llama-2-7b-chat-hf")
parser.add_argument('--model',
- choices=["kaist-ai/prometheus-13b-v1.0", "kaist-ai/prometheus-7b-v1.0"],
- default="kaist-ai/prometheus-13b-v1.0",
+ choices=["prometheus-eval/prometheus-13b-v1.0", "prometheus-eval/prometheus-7b-v1.0"],
+ default="prometheus-eval/prometheus-13b-v1.0",
help="Model to use for rubric evaluation.")
args = parser.parse_args()
diff --git a/eval/evaluation_prometheus.py b/eval/evaluation_prometheus.py
index 3f20a9c..2569bd9 100644
--- a/eval/evaluation_prometheus.py
+++ b/eval/evaluation_prometheus.py
@@ -171,9 +171,9 @@ if __name__ == "__main__":
parser.add_argument('--tokenizer', default="meta-llama/Llama-2-7b-chat-hf")
parser.add_argument('--model',
- choices=["kaist-ai/prometheus-13b-v1.0", "kaist-ai/prometheus-7b-v1.0"],
- default="kaist-ai/prometheus-13b-v1.0",
- help="Model to use; options are 'kaist-ai/prometheus-13b-v1.0' or 'kaist-ai/prometheus-7b-v1.0'")
+ choices=["prometheus-eval/prometheus-13b-v1.0", "prometheus-eval/prometheus-7b-v1.0"],
+ default="prometheus-eval/prometheus-13b-v1.0",
+ help="Model to use; options are 'prometheus-eval/prometheus-13b-v1.0' or 'prometheus-eval/prometheus-7b-v1.0'")
parser.add_argument('--disable_sample', action='store_true', help='Whether to disable sampling; default is False')
parser.add_argument('--temperature', type=float, default=0.01, help='Temperature for generation; default is 0.01')
parser.add_argument('--top_p', type=float, default=0.95, help='Top P for generation; default is 0.95')