Skip to content

SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]

Support multilingual evaluation

Kabir Khandpuraae4a22f4fca
diff --git a/config/anthropic_filemap_ml.yaml b/config/anthropic_filemap_ml.yaml
new file mode 100644
index 00000000..e3cabeb9
--- /dev/null
+++ b/config/anthropic_filemap_ml.yaml
@@ -0,0 +1,72 @@
+# This template is heavily inspired by anthropic and openhands
+agent:
+ type: default
+ templates:
+ system_template: |-
+ You are a helpful assistant that can interact with a computer to solve tasks.
+ instance_template: |-
+ <uploaded_files>
+ {{working_dir}}
+ </uploaded_files>
+ I've uploaded a code repository in the directory {{working_dir}}. Consider the following PR description:
+
+ <pr_description>
+ {{problem_statement}}
+ </pr_description>
+
+ Can you help me implement the necessary changes to the repository so that the requirements specified in the <pr_description> are met?
+ I've already taken care of all changes to any of the test files described in the <pr_description>. This means you DON'T have to modify the testing logic or any of the tests in any way!
+ Your task is to make the minimal changes to non-tests files in the {{working_dir}} directory to ensure the <pr_description> is satisfied.
+ Follow these steps to resolve the issue:
+ 1. As a first step, it might be a good idea to find and read code relevant to the <pr_description>
+ 2. Create a script to reproduce the error and execute it using the bash tool, to confirm the error
+ 3. Edit the sourcecode of the repo to resolve the issue
+ 4. Rerun your reproduce script and confirm that the error is fixed!
+ 5. Think about edgecases and make sure your fix handles them as well
+ Your thinking should be thorough and so it's fine if it's very long.
+ next_step_template: |-
+ OBSERVATION:
+ {{observation}}
+ next_step_no_output_template: |-
+ Your command ran successfully and did not produce any output.
+ tools:
+ execution_timeout: 300
+ bundles:
+ - path: tools/ml_setup
+ - path: tools/registry
+ - path: tools/edit_anthropic
+ - path: tools/review_on_submit_m
+ - path: tools/diff_state
+ enable_bash_tool: true
+ parse_function:
+ type: function_calling
+ registry_variables:
+ USE_FILEMAP: 'true'
+ SUBMIT_REVIEW_MESSAGES:
+ - |
+ Thank you for your work on this issue. Please carefully follow the steps below to help review your changes.
+
+ 1. If you made any changes to your code after running the reproduction script, please run the reproduction script again.
+ If the reproduction script is failing, please revisit your changes and make sure they are correct.
+ If you have already removed your reproduction script, please ignore this step.
+ 2. Remove your reproduction script (if you haven't done so already).
+ 3. If you have modified any TEST files, please revert them to the state they had before you started fixing the issue.
+ You can do this with `git checkout -- /path/to/test/file`. Use below <diff> to find the files you need to revert.
+ 4. Run the submit command again to confirm.
+
+ Here is a list of all of your changes:
+
+ <diff>
+ {{diff}}
+ </diff>
+ history_processors:
+ - type: cache_control
+ last_n_messages: 2
+ model:
+ name: claude-3-7-sonnet-20250219
+ max_output_tokens: 64000
+ per_instance_cost_limit: 2.50
+ per_instance_call_limit: 150
+ total_cost_limit: 500.0
+ temperature: 0.0
+ delay: 0.0
diff --git a/tools/ml_setup/bin/do_nothing b/tools/ml_setup/bin/do_nothing
new file mode 100644
index 00000000..fc9d4b2f
--- /dev/null
+++ b/tools/ml_setup/bin/do_nothing
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+main() {
+ echo "Executed do_nothing"
+}
+
+main "$@"
diff --git a/tools/ml_setup/config.yaml b/tools/ml_setup/config.yaml
new file mode 100644
index 00000000..44404573
--- /dev/null
+++ b/tools/ml_setup/config.yaml
@@ -0,0 +1,5 @@
+# This exists to make the validation pass. The only useful part of the tool is install.sh
+tools:
+ do_nothing:
+ signature: "do_nothing"
+ docstring: "does nothing"
diff --git a/tools/ml_setup/install.sh b/tools/ml_setup/install.sh
new file mode 100644
index 00000000..81cb3af0
--- /dev/null
+++ b/tools/ml_setup/install.sh
@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+
+# Define variables to exclude
+EXCLUDE_VARS="PWD|LANG|PYTHONPATH|ROOT|PS0|PS1|PS2|_|OLDPWD|LC_ALL|LANG|LSCOLORS|SHLVL"
+
+
+echo "Original Environment Variables:"
+env | sort
+
+# Only add Python 3.11 to PATH if no python exists
+if ! command -v python &> /dev/null; then
+ echo -e "\nNo Python found in system, adding Python 3.11 to PATH"
+ export PATH="/root/python3.11/bin:$PATH"
+
+ # Create python/pip aliases
+ ln -s "/root/python3.11/bin/python3" "/root/python3.11/bin/python"
+ ln -s "/root/python3.11/bin/pip3" "/root/python3.11/bin/pip"
+ echo "Created symlinks: python -> python3, pip -> pip3"
+else
+ echo -e "\nPython already exists in system, skipping Python 3.11 setup"
+fi
+
+# Attempt to read and set process 1 environment
+echo -e "\nSetting environment variables from /proc/1/environ..."
+if [ -r "/proc/1/environ" ]; then
+ while IFS= read -r -d '' var; do
+ # Skip excluded variables
+ if ! echo "$var" | grep -qE "^(${EXCLUDE_VARS})="; then
+ # If the variable is PATH, append and deduplicate
+ if [[ "$var" =~ ^PATH= ]]; then
+ # Combine paths and remove duplicates while preserving order
+ export PATH="$(echo "${PATH}:${var#PATH=}" | tr ':' '\n' | awk '!seen[$0]++' | tr '\n' ':' | sed 's/:$//')"
+ else
+ export "$var"
+ fi
+ fi
+ done < /proc/1/environ
+ echo "Successfully imported environment from /proc/1/environ"
+else
+ echo "Cannot access /proc/1/environ - Permission denied"
+fi
+
+# Print updated environment variables
+echo -e "\nUpdated Environment Variables:"
+env | sort