SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]
Doc: Fix default config & model (#1105)
Kilian Lieret83fe102f1fb2
diff --git a/.devcontainer/bashrc_epilog.sh b/.devcontainer/bashrc_epilog.shindex 8f3f8855..172c8432 100644--- a/.devcontainer/bashrc_epilog.sh+++ b/.devcontainer/bashrc_epilog.sh@@ -6,7 +6,7 @@ fiecho "Here's an example SWE-agent command to try out:"echo "sweagent run \\- --agent.model.name=claude-3-5-sonnet-20241022 \\+ --agent.model.name=claude-3-7-sonnet-latest \\--agent.model.per_instance_cost_limit=2.00 \\--env.repo.github_url=https://github.com/SWE-agent/test-repo \\--problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1 \\diff --git a/config/250212_sweagent_heavy_sbl.yaml b/config/250212_sweagent_heavy_sbl.yamlindex a7c0c813..89470625 100644--- a/config/250212_sweagent_heavy_sbl.yaml+++ b/config/250212_sweagent_heavy_sbl.yaml@@ -9,7 +9,7 @@ agent:# +filemap- type: defaultmodel: &model- name: claude-3-5-sonnet-20241022+ name: claude-3-7-sonnet-latestapi_key: $CLAUDE_API_KEY_ROTATIONper_instance_cost_limit: 1.5per_instance_call_limit: 75diff --git a/docs/installation/keys.md b/docs/installation/keys.mdindex 5e7855ba..be27a7ac 100644--- a/docs/installation/keys.md+++ b/docs/installation/keys.md@@ -40,7 +40,7 @@ Here are a few options for `--agent.model.name`:| Model | API key | Comment || ----- | ------- | ------- |-| `claude-3-5-sonnet-20241022` | `ANTHROPIC_API_KEY` | Our recommended model |+| `claude-3-7-sonnet-latest` | `ANTHROPIC_API_KEY` | Our recommended model || `gpt-4o` | `OPENAI_API_KEY` | || `o1-preview` | `OPENAI_API_KEY` | You might need to set temperature and sampling to the supported values. |diff --git a/docs/reference/agent_config.md b/docs/reference/agent_config.mdindex 4f6e804b..de442a85 100644--- a/docs/reference/agent_config.md+++ b/docs/reference/agent_config.md@@ -6,7 +6,7 @@ To learn about the agent class itself, see the [agent class reference page](agenIt might be easiest to simply look at some of our example configurations in the [config dir](https://github.com/SWE-agent/SWE-agent/tree/main/config).<details>-<summary>Example: default config <code>default.yaml</code></summary>+<summary>Example: default config <code>anthropic_filemap.yaml</code></summary>```yaml--8<-- "config/default_from_url.yaml"diff --git a/docs/usage/batch_mode.md b/docs/usage/batch_mode.mdindex ea948f2f..e630c11e 100644--- a/docs/usage/batch_mode.md+++ b/docs/usage/batch_mode.md@@ -14,7 +14,7 @@ Let's run on three [SWE-bench](https://www.swebench.com/) issues which will be d```bashsweagent run-batch \- --config config/default.yaml \+ --config config/anthropic_filemap.yaml \--agent.model.name gpt-4o \--agent.model.per_instance_cost_limit 2.00 \--instances.type swe_bench \@@ -58,7 +58,7 @@ Let's speed things up and run on 5 instances at once. Only a single line to chan```bash hl_lines="4-4"sweagent run-batch \- --config config/default.yaml \+ --config config/anthropic_filemap.yaml \--agent.model.name gpt-4o \--num_workers 3 \--agent.model.per_instance_cost_limit 2.00 \@@ -88,7 +88,7 @@ thereby easing CPU pressure. Default is 0.3.```bashsweagent run-batch \- --config config/default.yaml \+ --config config/anthropic_filemap.yaml \--agent.model.name gpt-4o \--instances.type file \--instances.path instances.yaml \diff --git a/docs/usage/cl_tutorial.md b/docs/usage/cl_tutorial.mdindex 3d88c9bd..59049a7a 100644--- a/docs/usage/cl_tutorial.md+++ b/docs/usage/cl_tutorial.md@@ -103,13 +103,13 @@ But we can also split it up into multiple files and additional command line optigithub_url: https://github.com/SWE-agent/test-repo```-The default config file is `config/default.yaml`. Let's take a look at it:+The default config file is `config/anthropic_filemap.yaml`. Let's take a look at it:<details>-<summary>Example: default config <code>default.yaml</code></summary>+<summary>Example: default config <code>anthropic_filemap.yaml</code></summary>```yaml---8<-- "config/default_from_url.yaml"+--8<-- "config/anthropic_filemap.yaml"```</details>@@ -119,7 +119,7 @@ This file is also loaded when no other `--config` options are specified.So to make sure that we get the default templates in the above examples with `--config`, we should have added```bash---config config/default.yaml+--config config/anthropic_filemap.yaml```in addition to all the other `--config` options for the two examples above.@@ -207,7 +207,7 @@ In addition, you can also execute additional commands before starting the agent```bashsweagent run \- --agent.model.name=claude-3-5-sonnet-20241022 \+ --agent.model.name=claude-3-7-sonnet-latest \--env.post_startup_commands='["pip install flake8"]' \...```diff --git a/docs/usage/competitive_runs.md b/docs/usage/competitive_runs.mdindex 84c169da..fac58b1f 100644--- a/docs/usage/competitive_runs.md+++ b/docs/usage/competitive_runs.md@@ -14,7 +14,7 @@ We recently used two configurations for SWE-bench submissions* [250225_anthropic_filemap_simple_review.yaml](https://github.com/SWE-agent/SWE-agent/blob/main/config/250225_anthropic_filemap_simple_review.yaml):This is our current default one-attempt config. It uses `claude-3-7-sonnet-20250219`.* [250212_sweagent_heavy_sbl.yaml](https://github.com/SWE-agent/SWE-agent/blob/main/config/250212_sweagent_heavy_sbl.yaml):- This config runs 5 attempts with slightly different configurations using `claude-3-5-sonnet-20241022`,+ This config runs 5 attempts with slightly different configurations using `claude-3-7-sonnet-latest`,then uses o1 to discriminate between them.This is a very expensive configuration.If you use it, also make sure to use Claude 3.7 instead of claude 3.5.diff --git a/docs/usage/hello_world.md b/docs/usage/hello_world.mdindex 15b4a540..824abfb0 100644--- a/docs/usage/hello_world.md+++ b/docs/usage/hello_world.md@@ -20,7 +20,7 @@ Let's start with an absolutely trivial example and solve an issue about a simple```bashsweagent run \- --agent.model.name=claude-3-5-sonnet-20241022 \+ --agent.model.name=claude-3-7-sonnet-latest \--agent.model.per_instance_cost_limit=2.00 \--env.repo.github_url=https://github.com/SWE-agent/test-repo \--problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1diff --git a/docs/usage/hello_world_output.txt b/docs/usage/hello_world_output.txtindex 9d8f2744..b9b56755 100644--- a/docs/usage/hello_world_output.txt+++ b/docs/usage/hello_world_output.txt@@ -17,7 +17,7 @@๐ฑ DEBUG Resetting repository SWE-agent__test-repo to commit HEAD๐ INFO Running agent๐ค INFO Trajectory will be saved to- /Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-5-sonnet-20241022__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent__test-+ /Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-7-sonnet-latest__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent__test-repo-i1/SWE-agent__test-repo-i1.traj๐ฆ DEBUG Uploading file from /Users/fuchur/Documents/24/git_sync/SWE-agent/tools/registry to /root/tools/registry๐ฆ DEBUG Created zip file at /var/folders/g3/1y2_hl1j16scls01cxqm76s80000gn/T/tmpswoslekq/zipped_transfer.zip@@ -363,7 +363,7 @@(Current directory: /SWE-agent__test-repo)bash-$๐ค INFO Trajectory saved to- /Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-5-sonnet-20241022__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent__test-+ /Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-7-sonnet-latest__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent__test-repo-i1/SWE-agent__test-repo-i1.trajโญโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ Submission successful ๐ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎโ SWE-agent has produced a patch that it believes will solve the issue you submitted! โ@@ -371,7 +371,7 @@โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ# The patch has been saved to your local filesystem at:- PATCH_FILE_PATH='/Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-5-sonnet-20241022__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent+ PATCH_FILE_PATH='/Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-7-sonnet-latest__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agenttest-repo-i1.patch'# Inspect it:cat "${PATCH_FILE_PATH}"diff --git a/sweagent/run/run_batch.py b/sweagent/run/run_batch.pyindex 2b04cc7c..e37816f0 100644--- a/sweagent/run/run_batch.py+++ b/sweagent/run/run_batch.py@@ -16,7 +16,7 @@ sweagent run-batch \\--instances.split dev \\--instances.slice :50 \\ # first 50 instances--instances.shuffle=True \\ # shuffle instances (with fixed seed)- --config config/default.yaml \\ # configure model+ --config config/anthropic_filemap.yaml \\ # configure model--agent.model.name gpt-4o[/green]diff --git a/sweagent/run/run_single.py b/sweagent/run/run_single.pyindex 8ba4f77b..39b58e33 100644--- a/sweagent/run/run_single.py+++ b/sweagent/run/run_single.py@@ -11,7 +11,7 @@Basic usage: Run over a [bold][cyan]github issue[/bold][/cyan][green]:-sweagent run --config config/default.yaml --agent.model.name "gpt-4o" \\+sweagent run --config config/anthropic_filemap.yaml --agent.model.name "gpt-4o" \\--env.repo.github_url=https://github.com/SWE-agent/test-repo/ \\--problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1[/green]@@ -21,7 +21,7 @@ You can set the image with [green]--env.docker.image[/green].Here's an example that uses [bold][cyan]modal[/bold][/cyan] instead of docker and also a [bold][cyan]local repository[/bold][/cyan]:-[green]sweagent run --config config/default.yaml --agent.model.name "gpt-4o" \\+[green]sweagent run --config config/anthropic_filemap.yaml --agent.model.name "gpt-4o" \\--env.deployment.type=modal --env.repo.path /path/to/repo \\--problem_statement.path=path/to/problem_statement.md[/green]diff --git a/tests/test_data/trajectories/gpt4__swe-bench-dev-easy_first_only__default__t-0.00__p-0.95__c-3.00__install-1/args.yaml b/tests/test_data/trajectories/gpt4__swe-bench-dev-easy_first_only__default__t-0.00__p-0.95__c-3.00__install-1/args.yamlindex e3a9be65..672bea69 100644--- a/tests/test_data/trajectories/gpt4__swe-bench-dev-easy_first_only__default__t-0.00__p-0.95__c-3.00__install-1/args.yaml+++ b/tests/test_data/trajectories/gpt4__swe-bench-dev-easy_first_only__default__t-0.00__p-0.95__c-3.00__install-1/args.yaml@@ -493,7 +493,7 @@ agent:end_name: nullname: _constrain_linesignature: _constrain_line- config_file: config/default.yaml+ config_file: config/anthropic_filemap.yamlmodel:host_url: localhost:11434model_name: gpt4