Skip to content

SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]

Feat: sb-cli support for verified

Kilian Lieretca3cfb365a36
diff --git a/sweagent/run/hooks/swe_bench_evaluate.py b/sweagent/run/hooks/swe_bench_evaluate.py
index f3ed8a6b..67bdae27 100644
--- a/sweagent/run/hooks/swe_bench_evaluate.py
+++ b/sweagent/run/hooks/swe_bench_evaluate.py
@@ -17,7 +17,7 @@ from sweagent.utils.log import get_logger
class SweBenchEvaluate(RunHook):
- _SUBSET_MAP = {"lite": "swe-bench_lite"}
+ _SUBSET_MAP = {"lite": "swe-bench_lite", "verified": "swe-bench_verified"}
def __init__(self, output_dir: Path, subset: str, split: str, continuous_submission_every: int = 0) -> None:
super().__init__()