SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]
Remove docker folder; add docs (#907)
Kilian Lieret9b43290efcd5
diff --git a/.github/workflows/release-dockerhub-nightly.yaml b/.github/workflows/release-dockerhub-nightly.yamldeleted file mode 100644index 6c055647..00000000--- a/.github/workflows/release-dockerhub-nightly.yaml+++ /dev/null@@ -1,24 +0,0 @@-name: Release to dockerhub (nightly)--on:- workflow_dispatch:- schedule:- - cron: '0 0 * * *'- push:- tags:- - 'v*.*.*'--jobs:- release:- runs-on: ubuntu-latest- steps:- - name: Login to Docker Hub- uses: docker/login-action@v3- with:- username: sweagent- password: ${{ secrets.DOCKERHUB_TOKEN }}- - name: Checkout code- uses: actions/checkout@v4- - name: Run release script- if: (github.ref == 'refs/heads/main') && (github.repository == 'SWE-agent/SWE-agent')- run: ./release_dockerhub.sh sweagent latest\ No newline at end of filediff --git a/.github/workflows/release-dockerhub-release.yaml b/.github/workflows/release-dockerhub-release.yamldeleted file mode 100644index 07d1677d..00000000--- a/.github/workflows/release-dockerhub-release.yaml+++ /dev/null@@ -1,22 +0,0 @@-name: Release to dockerhub (release)--on:- push:- tags:- - 'v*.*.*'--jobs:- release:- runs-on: ubuntu-latest- steps:- - name: Login to Docker Hub- uses: docker/login-action@v3- with:- username: sweagent- password: ${{ secrets.DOCKERHUB_TOKEN }}- - name: Checkout code- uses: actions/checkout@v4- - name: Get release name- run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/v}" >> $GITHUB_ENV- - name: Run release script- run: ./release_dockerhub.sh sweagent ${{ env.RELEASE_VERSION }}\ No newline at end of filediff --git a/README.md b/README.mdindex 666b116a..2d747faf 100644--- a/README.md+++ b/README.md@@ -88,8 +88,6 @@ MIT. Check `LICENSE`.<div align="center">[](https://github.com/SWE-agent/SWE-agent/actions/workflows/pytest.yaml)-[](https://github.com/SWE-agent/SWE-agent/actions/workflows/release-dockerhub-nightly.yaml)-[](https://github.com/SWE-agent/SWE-agent/actions/workflows/release-dockerhub-release.yaml)[](https://github.com/SWE-agent/SWE-agent/actions/workflows/build-docs.yaml)[](https://codecov.io/gh/SWE-agent/SWE-agent)[](https://results.pre-commit.ci/latest/github/SWE-agent/SWE-agent/main)diff --git a/docker/README.md b/docker/README.mddeleted file mode 100644index c4773a76..00000000--- a/docker/README.md+++ /dev/null@@ -1,70 +0,0 @@-# Docker-To ensure reproducibility and sandboxed execution of SWE-agent actions across systems, we adopt practices established in [prior work](https://intercode-benchmark.github.io/) and use [🐋 Docker](https://www.docker.com/) containers to carry out SWE-agent inference.--* The `swe.Dockerfile` file is the customized image written for the environment of SWE-agent.-* The `./setup.sh` script automatically builds this image.-* When `run.py` is invoked, containers are automatically created from the built image.- * There is no need to manually build a container from the image.--Here, we explain what each line in `swe.Dockerfile` does:--1. **Base Image**: Start from the latest version of the Ubuntu image.-```bash-FROM ubuntu:latest-```-2. **Build Argument**: Define a build argument `MINICONDA_URL` that will be used to specify the Miniconda installer URL during the build process.-```bash-ARG MINICONDA_URL-```-3. **Install Third-Party Tools**: Update the package lists for the Ubuntu package manager and install several essential development tools. Clean up after the installation.-```bash-RUN apt-get update && \- apt-get install -y bash gcc git jq wget g++ make && \- apt-get clean && \- rm -rf /var/lib/apt/lists/*-```-4. **Initialize Git**: Configure global Git settings with a user email and name.-```bash-RUN git config --global user.email "[email protected]"-RUN git config --global user.name "sweagent"-```-5. **Environment Variables**: Set the `ROOT` environment variable and customize the shell prompt.-```bash-ENV ROOT='/dev/'-RUN prompt() { echo " > "; };-ENV PS1="> "-```-6. **Create Assets for Inference**: Create two files that are used to track metadata during an episode.-```bash-RUN touch /root/files_to_edit.txt-RUN touch /root/test.patch-```-7. **Enhance `ls` Command**: Modify the `.bashrc` file to alias the `ls` command.-```bash-RUN echo "alias ls='ls -F'" >> /root/.bashrc-```-8. Install Miniconda: Download and install Miniconda, then initialize conda with Bash support and add `conda-forge` to the channels list.-```bash-ENV PATH="/root/miniconda3/bin:${PATH}"-ARG PATH="/root/miniconda3/bin:${PATH}"-RUN wget ${MINICONDA_URL} -O miniconda.sh \- && mkdir /root/.conda \- && bash miniconda.sh -b \- && rm -f miniconda.sh-RUN conda --version \- && conda init bash \- && conda config --append channels conda-forge-```-9. **Install Python Packages**: Copy the `requirements.txt` file into the image and install the specified Python packages.-```bash-COPY docker/requirements.txt /root/requirements.txt-RUN pip install -r /root/requirements.txt-```-10. **Set Working Directory**: Set the working directory to the root directory.-```bash-WORKDIR /-```-11. **Default Command**: Set the default command to open a Bash shell when the container starts.-```bash-CMD ["/bin/bash"]-```\ No newline at end of filediff --git a/docker/ctf_requirements.txt b/docker/ctf_requirements.txtdeleted file mode 100644index c6ff0b51..00000000--- a/docker/ctf_requirements.txt+++ /dev/null@@ -1,8 +0,0 @@-pwntools-ipython-gmpy2-pycryptodome-mpmath-sympy-ruff-setproctitle\ No newline at end of filediff --git a/docker/enigma.Dockerfile b/docker/enigma.Dockerfiledeleted file mode 100644index 4e439bc0..00000000--- a/docker/enigma.Dockerfile+++ /dev/null@@ -1,78 +0,0 @@-FROM sweagent/swe-agent:0.6.1--ARG TARGETARCH--ENV DEBIAN_FRONTEND=noninteractive-RUN apt-get update && apt-get install -y \- build-essential \- vim \- cmake \- libgtk2.0-dev \- pkg-config \- libavcodec-dev \- libavformat-dev \- libswscale-dev \- libssl-dev \- libffi-dev \- libtbb2 \- libtbb-dev \- libjpeg-dev \- libpng-dev \- libtiff-dev \- ubuntu-desktop \- bc \- bsdmainutils \- tshark \- openjdk-17-jdk \- && rm -rf /var/lib/apt/lists/*--# Install radare2-WORKDIR /tmp-RUN apt-get update && apt-get install -y curl netcat qemu-user qemu-user-static-RUN curl -LO https://github.com/radareorg/radare2/releases/download/5.9.4/radare2_5.9.4_amd64.deb-RUN dpkg -i radare2_5.9.4_amd64.deb--# Sagemath-RUN apt-get update && apt-get install -y sagemath--# sqlmap and nikto-RUN apt-get install -y sqlmap nikto--# Install apktool and jadx-RUN apt-get install -y apktool-RUN wget https://github.com/skylot/jadx/releases/download/v1.4.7/jadx-1.4.7.zip-RUN unzip -d /usr/local jadx-1.4.7.zip-RUN rm -f jadx-1.4.7.zip--# Install wine & wine32-RUN dpkg --add-architecture i386 && apt update && apt install -y wine wine32--# Install ghidra-RUN cd /opt \- && wget -O ghidra.zip https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_11.0.1_build/ghidra_11.0.1_PUBLIC_20240130.zip \- && unzip ghidra.zip \- && rm -f ghidra.zip-ENV PATH=$PATH:/opt/ghidra_11.0.1_PUBLIC/support:/opt/ghidra_11.0.1_PUBLIC/Ghidra-COPY docker/ghidra_scripts /ghidra_scripts--# Install python requirements-ENV PIP_NO_CACHE_DIR=1-RUN pip install --upgrade pip-COPY docker/ctf_requirements.txt /root/requirements.txt-RUN pip install -r /root/requirements.txt && rm -f /root/requirements.txt--RUN cd /opt && git clone https://github.com/RsaCtfTool/RsaCtfTool.git && apt-get install -y libgmp3-dev libmpc-dev && cd RsaCtfTool && pip3 install -r "requirements.txt"-COPY docker/number_theory__fixed /opt/RsaCtfTool/lib/number_theory.py-ENV PATH=$PATH:/opt/RsaCtfTool-ENV PYTHONUNBUFFERED=1-ENV PWNLIB_NOTERM=1--# Install binwalk-RUN apt-get update && apt-get install -y binwalk--# Install sudo just in case the model still uses it-RUN apt-get install -y sudo--WORKDIR /--CMD ["/bin/bash"]\ No newline at end of filediff --git a/docker/getconda.sh b/docker/getconda.shdeleted file mode 100755index 5acf9929..00000000--- a/docker/getconda.sh+++ /dev/null@@ -1,20 +0,0 @@-#!/usr/bin/env bash--# Helper script to get the right conda version inside of the container-# This logic is put inside of the container rather than in the build script-# so that we can easily do multi-platform builds--arch=$1-echo "arch", $arch-if [[ "$arch" == "x86_64" || "$arch" == "amd64" ]]; then- echo "Building the x86 Docker image"- wget https://repo.anaconda.com/miniconda/Miniconda3-py310_24.1.2-0-Linux-x86_64.sh -O miniconda.sh-elif [[ "$arch" == "aarch64" || "$arch" == "arm64" ]]; then- echo "Ayy, arm64 in the house!"- wget https://repo.anaconda.com/miniconda/Miniconda3-py310_24.1.2-0-Linux-aarch64.sh -O miniconda.sh-else- echo "unknown architecture detected?"- echo $arch- exit 1-fi-echo "miniconda.sh downloaded"\ No newline at end of filediff --git a/docker/ghidra_scripts/Decompile.java b/docker/ghidra_scripts/Decompile.javadeleted file mode 100644index a4b4258c..00000000--- a/docker/ghidra_scripts/Decompile.java+++ /dev/null@@ -1,58 +0,0 @@-import java.io.File;-import java.io.IOException;-import java.util.ArrayList;-import java.util.List;--import org.apache.logging.log4j.LogManager;-import org.apache.logging.log4j.Logger;--import ghidra.app.script.GhidraScript;-import ghidra.app.util.Option;-import ghidra.app.util.OptionException;-import ghidra.app.util.exporter.CppExporter;-import ghidra.app.util.exporter.ExporterException;--public class Decompile extends GhidraScript {- private static Logger log;-- public Decompile() {- log = LogManager.getLogger(Decompile.class);- }-- public void export(String filename) {- File outputFile = new File(filename);- CppExporter cppExporter = new CppExporter();- cppExporter.setExporterServiceProvider(state.getTool());-- List<Option> options = new ArrayList<Option>();- Option cppExportHeaderOption =- new Option(CppExporter.CREATE_HEADER_FILE, Boolean.FALSE);- options.add(cppExportHeaderOption);- try {- cppExporter.setOptions(options);- } catch (OptionException e) {- log.error("Unable to set cppExporter options", e);- return;- }-- try {- cppExporter.export(outputFile, currentProgram, null, monitor);- } catch (IOException e) {- log.error("Failed writing decompiled code as output", e);- } catch (ExporterException e) {- log.error("Failed to export with cppExporter", e);- }- }-- @Override- public void run() throws Exception {- // IngredientDescription[] ingredients = getIngredientDescriptions();- // for (IngredientDescription ingredient : ingredients) {- // state.addParameter(ingredient.getID(), ingredient.getLabel(),- // ingredient.getType(), ingredient.getDefaultValue());- // }-- String[] args = getScriptArgs();- export(args[0]);- }-}\ No newline at end of filediff --git a/docker/ghidra_scripts/DecompileToJson.java b/docker/ghidra_scripts/DecompileToJson.javadeleted file mode 100644index cbcc5ded..00000000--- a/docker/ghidra_scripts/DecompileToJson.java+++ /dev/null@@ -1,79 +0,0 @@-import java.io.File;-import java.io.FileWriter;-import java.io.IOException;-import java.util.HashMap;--import org.apache.logging.log4j.LogManager;-import org.apache.logging.log4j.Logger;--import ghidra.app.script.GhidraScript;-import ghidra.app.decompiler.DecompInterface;-import ghidra.app.decompiler.DecompileResults;--import com.google.gson.*;--public class DecompileToJson extends GhidraScript {- private static Logger log;-- public DecompileToJson() {- log = LogManager.getLogger(DecompileToJson.class);- }-- public String find_main(HashMap<String, String> function_map) {- // Find the `main` function from the `__libc_start_main` call- if (!function_map.containsKey("entry")) return null;- String entry = function_map.get("entry");- int libc = entry.indexOf("__libc_start_main(");- if (libc == -1) return null;- int funcend = entry.indexOf(',', libc);- if (funcend == -1) return null;- return entry.substring(libc + 18, funcend);- }-- public void export(String filename) {- Gson gson = new GsonBuilder().setPrettyPrinting().create();- File outputFile = new File(filename);- String main_func = null;- HashMap<String, String> function_map = new HashMap<String, String>();- HashMap<String, String> address_map = new HashMap<String, String>();- DecompInterface ifc = new DecompInterface();- ifc.openProgram(currentProgram);-- for (var func : currentProgram.getListing().getFunctions(Boolean.TRUE)) {- DecompileResults res = ifc.decompileFunction(func,0,monitor);- if (!res.decompileCompleted()) {- System.err.println(res.getErrorMessage());- continue;- }- String code = res.getDecompiledFunction().getC();- function_map.put(func.getName(), code);- address_map.put(func.getEntryPoint().toString(), func.getName());- System.out.println(func.getName());- }-- if (!function_map.containsKey("main"))- main_func = find_main(function_map);-- //HashMap<String, HashMap<String, String>> json_data = new HashMap<String, HashMap<String, String>>();- HashMap<String, Object> json_data = new HashMap<String, Object>();- json_data.put("functions", function_map);- json_data.put("addresses", address_map);- if (main_func != null)- json_data.put("main", main_func);- // Convert the HashMap to JSON- String json = gson.toJson(json_data);-- // Write JSON to file- try (FileWriter writer = new FileWriter(outputFile)) {- writer.write(json);- } catch (IOException e) {- e.printStackTrace();- }- }-- @Override- public void run() throws Exception {- String[] args = getScriptArgs();- export(args[0]);- }-}diff --git a/docker/ghidra_scripts/Disassemble.java b/docker/ghidra_scripts/Disassemble.javadeleted file mode 100644index 4988a1c9..00000000--- a/docker/ghidra_scripts/Disassemble.java+++ /dev/null@@ -1,34 +0,0 @@-import java.io.File;-import java.io.IOException;--import org.apache.logging.log4j.LogManager;-import org.apache.logging.log4j.Logger;--import ghidra.app.script.GhidraScript;-import ghidra.app.util.exporter.AsciiExporter;--public class Disassemble extends GhidraScript {- private static Logger log;-- public Disassemble() {- log = LogManager.getLogger(Disassemble.class);- }-- public void export(String filename) {- File outputFile = new File(filename);- AsciiExporter asciiExporter = new AsciiExporter();- asciiExporter.setExporterServiceProvider(state.getTool());-- try {- asciiExporter.export(outputFile, currentProgram, null, monitor);- } catch (IOException e) {- log.error("Failed writing disassembled code as output", e);- }- }-- @Override- public void run() throws Exception {- String[] args = getScriptArgs();- export(args[0]);- }-}\ No newline at end of filediff --git a/docker/ghidra_scripts/DisassembleToJson.java b/docker/ghidra_scripts/DisassembleToJson.javadeleted file mode 100644index 214dff01..00000000--- a/docker/ghidra_scripts/DisassembleToJson.java+++ /dev/null@@ -1,302 +0,0 @@-import java.io.File;-import java.io.FileWriter;-import java.io.IOException;-import java.util.List;-import java.util.HashMap;--import org.apache.logging.log4j.LogManager;-import org.apache.logging.log4j.Logger;--import ghidra.app.script.GhidraScript;-import ghidra.app.util.EolComments;-import ghidra.program.model.listing.CodeUnit;-import ghidra.program.model.listing.CodeUnitFormat;-import ghidra.program.model.listing.CodeUnitFormatOptions;-import ghidra.program.model.listing.CodeUnitIterator;-import ghidra.program.model.listing.Data;-import ghidra.program.model.listing.Instruction;-import ghidra.program.model.listing.Listing;-import ghidra.program.model.listing.Variable;-import ghidra.program.model.mem.MemoryAccessException;-import ghidra.program.model.symbol.Symbol;-import ghidra.app.util.template.TemplateSimplifier;-import ghidra.app.util.viewer.field.EolExtraCommentsOption;--import com.google.gson.*;--public class DisassembleToJson extends GhidraScript {- private static Logger log;-- public DisassembleToJson() {- log = LogManager.getLogger(DisassembleToJson.class);- }-- private String getBytes(CodeUnit cu) {- StringBuffer bytesbuf = new StringBuffer();- try {- byte[] bytes;- if (cu instanceof Instruction instr) {- bytes = instr.getParsedBytes();- }- else {- bytes = cu.getBytes();- }- for (int i = 0; i < bytes.length; ++i) {- if (bytes[i] >= 0x00 && bytes[i] <= 0x0F) {- bytesbuf.append("0");- }- bytesbuf.append(Integer.toHexString(bytes[i] & 0xff));- }- }- catch (MemoryAccessException e) {- return "";- }- return bytesbuf.toString();- }-- private String getOperands(CodeUnit cu, CodeUnitFormat cuFormat) {- StringBuffer buffy = new StringBuffer();- if (cu instanceof Instruction) {-- Instruction inst = (Instruction) cu;-- int opCnt = inst.getNumOperands();- String firstSeparator = ((Instruction) cu).getSeparator(0);-- String[] opSeparators = new String[opCnt];- String[] opReps = new String[opCnt];- for (int i = 0; i < opCnt; ++i) {- opReps[i] = cuFormat.getOperandRepresentationString(cu, i);- opSeparators[i] = ((Instruction) cu).getSeparator(i + 1);- if (opSeparators[i] == null) {- opSeparators[i] = "";- }- }-- if (firstSeparator != null) {- buffy.append(firstSeparator);- }- for (int i = 0; i < opCnt; ++i) {- buffy.append(opReps[i]);- buffy.append(opSeparators[i]);- }- }- else if (cu instanceof Data) {- Data data = (Data) cu;- String opRep = cuFormat.getDataValueRepresentationString(data);- buffy.append(opRep);- }- return buffy.toString();- }-- private String getVariableSorageString(Variable var) {- String offsetStr;- if (var.isStackVariable()) {- int offset = var.getStackOffset();- offsetStr =- (offset >= 0 ? " 0x" + Integer.toHexString(offset) : "-0x" + Integer.toHexString(-offset));- }- else if (var.isRegisterVariable()) {- offsetStr = var.getRegister().getName();- }- else {- offsetStr = var.getVariableStorage().toString();- }- return offsetStr;- }-- public String find_main(HashMap<String, String> function_map) {- // Find the `main` function from the `__libc_start_main` call- if (!function_map.containsKey("entry")) return null;- String entry = function_map.get("entry");- int libc = entry.indexOf("CALL <EXTERNAL>::__libc_start_main");- if (libc == -1) {- // Does not have symbol for __libc_start_main- // These two instructions are checked:- // PUSH <func-ptr>- // CALL <start-func>- // The <func-ptr> is the main function.- libc = entry.indexOf("CALL FUN_");- if (libc == -1) return null;- entry = entry.substring(0, libc);- int push = entry.lastIndexOf("PUSH FUN_");- if (push == -1) return null;- int funcend = entry.indexOf('\n', push);- return entry.substring(push + 12, funcend);- } else {- // Has symbol for __libc_start_main- // These two instructions are checked:- // MOV RDI,<func-ptr>- // CALL EXTERNAL::__libc_start_main- // The <func-ptr> is the main function.- entry = entry.substring(0, libc);- int rdi = entry.lastIndexOf("MOV RDI,");- if (rdi == -1) return null;- int funcend = entry.indexOf('\n', rdi);- if (funcend == -1) return null;- return entry.substring(rdi + 16, funcend);- }- }-- public void export(String filename) {- Gson gson = new GsonBuilder().setPrettyPrinting().create();- File outputFile = new File(filename);- HashMap<String, String> function_map = new HashMap<String, String>();- HashMap<String, String> address_map = new HashMap<String, String>();- String main_func = null;-- // Formatter for disassembled code- TemplateSimplifier simplifier = new TemplateSimplifier();- simplifier.setEnabled(false);- CodeUnitFormatOptions formatOptions = new CodeUnitFormatOptions(- CodeUnitFormatOptions.ShowBlockName.NEVER,- CodeUnitFormatOptions.ShowNamespace.NON_LOCAL,- null,- true, // doRegVariableMarkup- true, // doStackVariableMarkup- true, // includeInferredVariableMarkup- true, // alwaysShowPrimaryReference- true, // includeScalarReferenceAdjustment- true, // showLibraryInNamespace- true, // followReferencedPointers- simplifier- );- var cuf = new CodeUnitFormat(formatOptions);-- // Annoying thing: for some reason catch blocks are not considered part of- // a function (listing.getFunctionContaining() returns null), so we'll just- // keep track of the last function we saw and assume that the next code unit- // belongs to the same function if it's not in a function.-- ghidra.program.model.listing.Function lastFunc = null;- CodeUnitIterator cuIterator = currentProgram.getListing().getCodeUnits(true);- Listing listing = currentProgram.getListing();- for (var cu : cuIterator) {- var currentAddress = cu.getMinAddress();- var func = listing.getFunctionContaining(currentAddress);- if (func == null) {- if (cu instanceof Instruction && lastFunc != null) {- log.warn(String.format(- "Instruction at [%s] not in a function; assuming it's part of %s",- currentAddress, lastFunc- ));- func = lastFunc;- }- else {- // System.err.printf(- // "[%s] Note: skipping CU of type %s\n",- // currentAddress, cu.getClass().getName()- // );- continue;- }- }- lastFunc = func;- boolean isFunctionEntryPoint = func.getEntryPoint().equals(currentAddress);- // Get the code for this function so far, or create a new one- String code = function_map.getOrDefault(func.getName(), "");-- // If this is the first code unit in the function, add preamble- if (isFunctionEntryPoint) {-- log.info(String.format(- "Processing: %s [%s,%s]",- func.getName(),- func.getBody().getMinAddress(),- func.getBody().getMaxAddress()- ));- // Add comment with function signature- code += String.format("; %s\n", func.getPrototypeString(true, true));- // Add comment with parameters- code += "; Parameters:\n";- for (var v : func.getParameters()) {- code += String.format("; %-14s %-14s %s\n",- v.getName(), v.getDataType().getDisplayName(), getVariableSorageString(v)- );- }- // Add comment with local variables on the stack- code += "; Stack variables:\n";- for (var v : func.getLocalVariables()) {- code += String.format("; %-14s %-14s %s\n",- v.getName(), v.getDataType().getDisplayName(), getVariableSorageString(v)- );- }- }- String addrString = cu.getAddressString(true, false);- // Any pre-comment- String preComment = cu.getComment(CodeUnit.PRE_COMMENT);- if (preComment != null) {- code += String.format("%-16s %-16s ; %s\n",- "",- "",- preComment- );- }-- // Primary symbol name (either function name or label)- Symbol primarySymbol = cu.getPrimarySymbol();- if (primarySymbol != null) {- code += String.format("%-16s %-16s %s:\n", "", "", primarySymbol.getName());- }-- // Disassembly- code += String.format("%-16s %-16s %-11s %-40s",- addrString,- getBytes(cu),- cu.getMnemonicString(),- getOperands(cu, cuf),- cuf.getRepresentationString(cu, true)- );-- // EOL comments- EolExtraCommentsOption eolOption = new EolExtraCommentsOption();- EolComments eolComments = new EolComments(cu, true, 6 /* arbitrary */, eolOption);- List<String> comments = eolComments.getComments();- if (comments.size() > 0) {- code += " ; " + String.join(", ", comments);- }-- // Trim trailing whitespace- code = code.stripTrailing();- code += "\n";-- // Any post-comment- String postComment = cu.getComment(CodeUnit.POST_COMMENT);- if (postComment != null) {- code += String.format("%-16s %-16s ; %s\n",- "",- "",- postComment- );- }-- // Update the code for this function- function_map.put(func.getName(), code);- // Add the address to map- address_map.put(func.getEntryPoint().toString(), func.getName());- }-- if (!function_map.containsKey("main"))- main_func = find_main(function_map);-- HashMap<String, Object> json_data = new HashMap<String, Object>();- json_data.put("functions", function_map);- json_data.put("addresses", address_map);- if (main_func != null)- json_data.put("main", main_func);- // Convert the HashMap to JSON- String json = gson.toJson(json_data);-- // Write JSON to file- try (FileWriter writer = new FileWriter(outputFile)) {- writer.write(json);- } catch (IOException e) {- e.printStackTrace();- }- }-- @Override- public void run() throws Exception {- String[] args = getScriptArgs();- export(args[0]);- }-}diff --git a/docker/number_theory__fixed b/docker/number_theory__fixeddeleted file mode 100644index 5233f8b3..00000000--- a/docker/number_theory__fixed+++ /dev/null@@ -1,675 +0,0 @@-#!/usr/bin/env python3-from __future__ import annotations--import logging-import math-import random-from functools import cache--logger = logging.getLogger("global_logger")--try:- import gmpy2 as gmpy-- gmpy_version = 2- mpz = gmpy.mpz- logger.info("[+] Using gmpy version 2 for math.")-except ImportError:- try:- import gmpy-- gmpy_version = 1- mpz = gmpy.mpz- logger.info("[+] Using gmpy version 1 for math.")- except ImportError:- gmpy_version = 0- mpz = int- gmpy = None- logger.warning(- "[!] Using native python functions for math, which is slow. install gmpy2 with: 'python3 -m pip install <module>'."- )---@cache-def list_prod(list_):- list_ = tuple(list_)- if (l := len(list_)) == 0:- return 1- return list_prod(list_[: l - 1]) * list_[-1]---digit_sum = lambda n: sum(int(d) for d in str(n))-A007814 = lambda n: (~n & n - 1).bit_length()-A135481 = lambda n: (~n & n - 1)-A000265 = lambda n: n // (A135481(n) + 1)---@cache-def mulmod(a, b, m):- if b == 0:- return 0- if b == 1:- return a % m- if b & 1 == 0:- return mulmod((a << 1) % m, b >> 1, m)- else:- return (a + mulmod(a, b - 1, m)) % m---def getpubkeysz(n):- if (size := n.bit_length()) & 1 != 0:- size += 1- return size---is_pow2 = lambda n: n & (n - 1) == 0---def _gcdext(a, b):- if a == 0:- return [b, 0, 1]- d, r = divmod(b, a)- g, y, x = _gcdext(r, a)- return [g, x - d * y, y]---def _isqrt(n):- if n == 0:- return 0- x, y = n, (n + 1) >> 1- while y < x:- x, y = y, (y + n // y) >> 1- return x---def _isqrt_rem(n):- i2 = _isqrt(n)- return i2, n - (i2 * i2)---def _isqrt_rem_gmpy(n):- i2 = _isqrt_gmpy(n)- return i2, n - (i2 * i2)---def _gcd(a, b):- while b:- a, b = b, a % b- return abs(a)---def _remove(n, p):- r = n- c = 0- while r % p == 0:- r //= p- c += 1- return r, c---def _introot(n, r=2):- if n < 0:- return None if r & 1 == 0 else -_introot(-n, r)- if n < 2:- return n- if r == 2:- return _isqrt(n)- lower, upper = 0, n- while lower != upper - 1:- mid = lower + ((upper - lower) >> 1)- m = pow(mid, r)- if m == n:- return mid- lower = mid * (m < n) + lower * (m >= n)- upper = mid * (m > n) + upper * (m <= n)- return lower---def _iroot(n, p):- b = introot(n, p)- return b, b**p == n---def _introot_gmpy(n, r=2):- if n < 0:- return None if r & 1 == 0 else -_introot_gmpy(-n, r)- return gmpy.root(n, r)[0]---def _introot_gmpy2(n, r=2):- if n < 0:- return None if r & 1 == 0 else -_introot_gmpy2(-n, r)- return gmpy.iroot(n, r)[0]---def _invmod(a, m):- a, x, u = a % m, 0, 1- while a:- x, u, m, a = u, x - (m // a) * u, a, m % a- return x---def _is_square(n):- if (h := n & 0xF) > 9 or h in [2, 3, 5, 6, 7, 8]:- return False- t = _isqrt(n)- return t * t == n---def _powmod_base_list(base_lst, exp, mod):- return list(powmod(i, exp, mod) for i in base_lst)---def _powmod_exp_list(base, exp_lst, mod):- return list(powmod(base, i, mod) for i in exp_lst)---def miller_rabin(n, k=40):- """ "- Taken from https://gist.github.com/Ayrx/5884790- Implementation uses the Miller-Rabin Primality Test- The optimal number of rounds for this test is 40- See http://stackoverflow.com/questions/6325576/how-many-iterations-of-rabin-miller-should-i-use-for-cryptographic-safe-primes- for justification- """-- if n == 2:- return True- if (n & 1 == 0) or (digit_sum(n) % 9 in [0, 3, 6]):- return False-- r, s = 0, n - 1- while s & 1 == 0:- r += 1- s >>= 1- i = 0- for _ in range(0, k):- a = random.randrange(2, n - 1)- if (x := pow(a, s, n)) in [1, n - 1]:- continue- j = 0- while j <= r - 1:- if (x := pow(x, 2, n)) == (n - 1):- break- j += 1- else:- return False- return True---def _fermat_prime_criterion(n, b=2):- """Fermat's prime criterion- Returns False if n is definitely composite, True if possible prime."""- return pow(b, n - 1, n) == 1---def _is_prime(n):- """- If fermats prime criterion is false by short circuit we dont need to keep testing bases, so we return false for a guaranteed composite.- Otherwise we keep trying with primes 3 and 5 as base. The sweet spot is primes 2,3,5, it doesn't improvee the runing time adding more primes to test as base.- If all the previous tests pass then we try with rabin miller.- All the tests are probabilistic.- """- if all(- (- _fermat_prime_criterion(n),- _fermat_prime_criterion(n, b=3),- _fermat_prime_criterion(n, b=5),- )- ):- return miller_rabin(n)- else:- return False---def _next_prime(n):- while True:- if _is_prime(n):- return n- n += 1---def erathostenes_sieve(n):- """- Returns a list of primes < n- """- sieve = [True] * n- for i in range(3, isqrt(n) + 1, 2):- if sieve[i]:- sieve[pow(i, 2) :: (i << 1)] = [False] * ((n - pow(i, 2) - 1) // (i << 1) + 1)- return [2] + [i for i in range(3, n, 2) if sieve[i]]---_primes = erathostenes_sieve---def _primes_yield(n):- p = i = 1- while i <= n:- p = next_prime(p)- yield p- i += 1---def _primes_yield_gmpy(n):- p = i = 1- while i <= n:- p = gmpy.next_prime(p)- yield p- i += 1---def _fib(n):- a, b = 0, 1- i = 0- while i <= n:- a, b = b, a + b- i += 1- return a---def ilogb(x, b):- """- greatest integer l such that b**l < = x.- """- l = 0- while x >= b:- x /= b- l += 1- return l---_primes_gmpy = lambda n: list(_primes_yield_gmpy(n))-_isqrt_gmpy = lambda n: int(gmpy.sqrt(n))-_invert = lambda a, b: pow(a, b - 2, b)-_lcm = lambda x, y: (x * y) // _gcd(x, y)-_ilog2_gmpy = lambda n: int(gmpy.log2(n))-_ilog_gmpy = lambda n: int(gmpy.log(n))-_ilog2_math = lambda n: int(math.log2(n))-_ilog_math = lambda n: int(math.log(n))-_ilog10_math = lambda n: int(math.log10(n))-_ilog10_gmpy = lambda n: int(gmpy.log10(n))-_mod = lambda a, b: a % b-_mul = lambda a, b: a * b-_is_divisible = lambda n, p: n % p == 0-_is_congruent = lambda a, b, m: (a - b) % m == 0---def _powmod(b, e, m):- r = 1- b %= m- while e > 0:- r = ((r * b) % m) * (e & 1) + r * ((e + 1) & 1)- e >>= 1- b = (b * b) % m- return r---def _fac(n):- """- Factorial- """- tmp = 1- for m in range(n, 1, -1):- tmp *= m- return tmp---@cache-def _lucas(n):- if n == 0:- return 2- if n == 1:- return 1- return _lucas(n - 1) + _lucas(n - 2)---if gmpy_version > 0:- gcd = gmpy.gcd- gcdext = gmpy.gcdext- is_square = gmpy.is_square- next_prime = gmpy.next_prime- is_prime = gmpy.is_prime- fib = gmpy.fib- primes = _primes_gmpy- lcm = gmpy.lcm- invert = gmpy.invert- invmod = gmpy.invert- remove = gmpy.remove- fac = gmpy.fac- if gmpy_version == 2:- iroot = gmpy.iroot- ilog = _ilog_gmpy- ilog2 = _ilog2_gmpy- ilog10 = _ilog10_gmpy- log = gmpy.log- log2 = gmpy.log2- log10 = gmpy.log10- mod = gmpy.f_mod- mul = gmpy.mul- powmod = gmpy.powmod- isqrt_rem = gmpy.isqrt_rem- introot = _introot_gmpy2- is_divisible = gmpy.is_divisible- is_congruent = gmpy.is_congruent- fdivmod = gmpy.f_divmod- lucas = gmpy.lucas- powmod_base_list = gmpy.powmod_base_list- powmod_exp_list = gmpy.powmod_exp_list- else:- iroot = gmpy.root- ilog = _ilog_math- ilog2 = _ilog2_math- ilog10 = _ilog10_math- log = math.log- log2 = math.log2- log10 = math.log10- mul = _mul- mod = _mod- powmod = pow- isqrt_rem = gmpy.sqrtrem- introot = _introot_gmpy- is_divisible = _is_divisible- is_congruent = _is_congruent- fdivmod = gmpy.fdivmod- lucas = _lucas- powmod_base_list = _powmod_base_list- powmod_exp_list = _powmod_exp_list-- isqrt = gmpy.isqrt-else:- remove = _remove- iroot = _iroot- gcd = _gcd- isqrt = _isqrt- isqrt_rem = _isqrt_rem- introot = _introot- invmod = _invmod- gcdext = _gcdext- is_square = _is_square- next_prime = _next_prime- fib = _fib- primes = erathostenes_sieve- is_prime = _is_prime- fib = _fib- primes = _primes- lcm = _lcm- invert = _invmod- powmod = _powmod- ilog = _ilog_math- ilog2 = _ilog2_math- ilog10 = _ilog10_math- log = math.log- log2 = math.log2- log10 = math.log10- mod = _mod- mul = _mul- is_divisible = _is_divisible- is_congruent = _is_congruent- fac = _fac- fdivmod = divmod- lucas = _lucas- powmod_base_list = _powmod_base_list- powmod_exp_list = _powmod_exp_list--legendre = lambda a, p: powmod(a, (p - 1) >> 1, p)-cuberoot = lambda n: introot(n, 3)---def factor_ned_probabilistic(n, e, d):- """- 800-56B R1 Recommendation for Pair-Wise Key Establishment Schemes Using Integer Factorization Cryptography in Appendix C.- """- n1, k = n - 1, d * e - 1- if k & 1 == 1:- return None- t, r = 0, k- while r & 1 == 0:- r >>= 1- t += 1- for _ in range(1, 101):- g = random.randint(0, n1)- if (y := pow(g, r, n)) == 1 or y == n1:- continue- for _ in range(1, t):- if (x := pow(y, 2, n)) == 1:- p = gcd(y - 1, n)- return p, n // p- if x == n1:- continue- y = x- if (x := pow(y, 2, n)) == 1:- p = gcd(x - 1, n)- return p, n // p---def trivial_factorization_with_n_b(n, b):- if (b2n4 := (b * b) - (n << 2)) > 0:- i = isqrt(b2n4)- p, q = int((b - i) >> 1), int((b + i) >> 1)- if p * q == n:- return p, q---def factor_ned_deterministic(n, e, d):- """- 800-56B R2 Recommendation for Pair-Wise Key Establishment Schemes Using Integer Factorization Cryptography in Appendix C.2.- """- k = d * e - 1- m, r = divmod(k * gcd(n - 1, k), n)- return trivial_factorization_with_n_b(n, ((n - r) // (m + 1)) + 1)---factor_ned = factor_ned_deterministic---trivial_factorization_with_n_phi = lambda n, phi: trivial_factorization_with_n_b(n, n - phi + 1)---def neg_pow(a, b, n):- """- Calculates a^{b} mod n when b is negative- """- assert b < 0- assert gcd(a, n) == 1- res = int(invert(a, n))- return powmod(res, b * (-1), n)---def common_modulus_related_message(e1, e2, n, c1, c2):- """- e1 --> Public Key exponent used to encrypt message m and get ciphertext c1- e2 --> Public Key exponent used to encrypt message m and get ciphertext c2- n --> Modulus- The following attack works only when m^{GCD(e1, e2)} < n- """-- g, a, b = gcdext(e1, e2)-- if g == 1:- return None-- c1 = neg_pow(c1, a, n) if a < 0 else powmod(c1, a, n)- c2 = neg_pow(c2, b, n) if a < 0 else powmod(c2, b, n)- ct = c1 * c2 % n- return int(introot(ct, g))---def phi(n, factors):- """- Euler totient function- """- if is_prime(n):- return n - 1- elif is_square(n):- i2 = isqrt(n)- return phi(i2, factors) * i2- else:- y = n- for p in factors:- if n % p == 0:- y //= p- y *= p - 1- n, _ = remove(n, p)- if n > 1:- y //= n- y *= n - 1- return y---def chinese_remainder(m, a):- S = 0- N = list_prod(tuple(m))- for i in range(0, len(m)):- Ni = N // m[i]- S += Ni * invert(Ni, m[i]) * a[i]- return S % N---def tonelli(n, p):- """- tonelli-shanks modular squareroot algorithm- """- assert legendre(n, p) == 1, "not a square (mod p)"- q = p - 1- q >>= (s := A007814(q))- if s == 1:- return powmod(n, (p + 1) >> 2, p)- for z in range(2, p):- if p - 1 == legendre(z, p):- break- c, r, t, m = powmod(z, q, p), powmod(n, (q + 1) >> 1, p), powmod(n, q, p), s- while (t - 1) % p != 0:- t2 = powmod(t, 2, p)- for i in range(1, m):- if (t2 - 1) % p == 0:- break- t2 = powmod(t2, 2, p)- b = powmod(c, 1 << (m - i - 1), p)- # r = (r * b) % p- r = mulmod(r, b, p)- c = powmod(b, 2, p)- # t = (t * c) % p- t = mulmod(t, c, p)- m = i- return r---def is_cube(n):- b = False- if (n % 9) in [0, 1, 8]:- a, b = iroot(n, 3)- return b---def dlp_bruteforce(g, h, p):- """- Try to solve the discrete logarithm problem:- x for g^x == h (mod p) with brute force.- """- for x in range(1, p):- if h == powmod(g, x, p):- return x---def rational_to_contfrac(x, y):- """Rational_to_contfrac implementation"""- a = x // y- if a * y == x:- return [a]- pquotients = rational_to_contfrac(y, x - a * y)- pquotients.insert(0, a)- return pquotients---def contfrac_to_rational(frac):- """Contfrac_to_rational implementation"""- if len(frac) == 0:- return (0, 1)- elif len(frac) == 1:- return (frac[0], 1)- else:- remainder = frac[1:]- (num, denom) = contfrac_to_rational(remainder)- return (frac[0] * num + denom, num)---def convergents_from_contfrac(frac, progress=False):- """Convergents_from_contfrac implementation"""- return [contfrac_to_rational(frac[:i]) for i in range(0, len(frac))]---def inv_mod_pow_of_2(factor, bit_count):- """- its orders of magnitude faster than invert(a, 2^k)- code borrowed from: https://algassert.com/post/1709- """- rest = factor & -2- acc = 1- for i in range(bit_count):- acc -= (acc & (1 << i)) * (rest << i)- mask = (1 << bit_count) - 1- return acc & mask---def mlucas(v, a, n):- """Helper function for williams_pp1(). Multiplies along a Lucas sequence modulo n."""- v1, v2 = v, (v * v - 2) % n- while a > 0:- v1, v2 = ((v1 * v1 - 2) % n, (v1 * v2 - v) % n) if a & 1 == 0 else ((v1 * v2 - v) % n, (v2 * v2 - 2) % n)- a >>= 1- return v1---__all__ = [- getpubkeysz,- gcd,- isqrt,- introot,- invmod,- gcdext,- is_square,- is_cube,- next_prime,- is_prime,- fib,- primes,- lcm,- invert,- powmod,- ilog2,- ilog,- ilog10,- mod,- log,- log2,- log10,- trivial_factorization_with_n_phi,- factor_ned,- neg_pow,- common_modulus_related_message,- phi,- list_prod,- chinese_remainder,- ilogb,- mul,- cuberoot,- isqrt_rem,- is_divisible,- is_congruent,- iroot,- dlp_bruteforce,- fac,- rational_to_contfrac,- contfrac_to_rational,- convergents_from_contfrac,- fdivmod,- inv_mod_pow_of_2,- mlucas,- lucas,- mulmod,- A000265,- powmod_base_list,- powmod_exp_list,- is_pow2,-]diff --git a/docker/requirements.txt b/docker/requirements.txtdeleted file mode 100644index 5333e8a1..00000000--- a/docker/requirements.txt+++ /dev/null@@ -1,6 +0,0 @@-anthropic[bedrock]-config-openai-tree-sitter==0.21.3 # See https://github.com/grantjenks/py-tree-sitter-languages/issues/64#issuecomment-2118048885.-tree-sitter-languages-uvdiff --git a/docker/swe.Dockerfile b/docker/swe.Dockerfiledeleted file mode 100644index 84aa3497..00000000--- a/docker/swe.Dockerfile+++ /dev/null@@ -1,54 +0,0 @@-FROM ubuntu:jammy--ARG TARGETARCH--# Install third party tools-RUN apt-get update && \- apt-get install -y bash gcc git jq wget g++ make && \- apt-get clean && \- rm -rf /var/lib/apt/lists/*--# Initialize git-RUN git config --global user.email "[email protected]"-RUN git config --global user.name "sweagent"--# Environment variables-ENV ROOT='/dev/'-RUN prompt() { echo " > "; };-ENV PS1="> "--# Create file for tracking edits, test patch-RUN touch /root/files_to_edit.txt-RUN touch /root/test.patch--# add ls file indicator-RUN echo "alias ls='ls -F'" >> /root/.bashrc--# Install miniconda-ENV PATH="/root/miniconda3/bin:${PATH}"-ARG PATH="/root/miniconda3/bin:${PATH}"-COPY SWE-agent/docker/getconda.sh .-RUN bash getconda.sh ${TARGETARCH} \- && rm getconda.sh \- && mkdir /root/.conda \- && bash miniconda.sh -b \- && rm -f miniconda.sh-RUN conda --version \- && conda init bash \- && conda config --append channels conda-forge--# Cache python versions-RUN conda create -y -n python3.9 python=3.9-RUN conda create -y -n python3.10 python=3.10--# Install python packages-COPY SWE-agent/docker/requirements.txt /root/requirements.txt-RUN pip install -r /root/requirements.txt--RUN mkdir /root/swe-rex-COPY swe-rex /root/swe-rex-RUN pip install -e /root/swe-rex--WORKDIR /--CMD ["/bin/bash"]diff --git a/docker/tiny_test.Dockerfile b/docker/tiny_test.Dockerfiledeleted file mode 100644index e35f55ba..00000000--- a/docker/tiny_test.Dockerfile+++ /dev/null@@ -1,15 +0,0 @@-FROM python:3.11.10-bullseye--ARG DEBIAN_FRONTEND=noninteractive-ENV TZ=Etc/UTC--WORKDIR /--RUN pip install pipx-RUN pipx install swe-rex-RUN pip install flake8-RUN pipx ensurepath--SHELL ["/bin/bash", "-c"]-# This is where pipx installs things-ENV PATH="$PATH:/root/.local/bin/"diff --git a/docker/tiny_test_x86.Dockerfile b/docker/tiny_test_x86.Dockerfiledeleted file mode 100644index 45d72d41..00000000--- a/docker/tiny_test_x86.Dockerfile+++ /dev/null@@ -1,15 +0,0 @@-FROM --platform=linux/x86_64 python:3.11.10-bullseye--ARG DEBIAN_FRONTEND=noninteractive-ENV TZ=Etc/UTC--WORKDIR /--RUN pip install pipx-RUN pipx install swe-rex-RUN pip install flake8-RUN pipx ensurepath--SHELL ["/bin/bash", "-c"]-# This is where pipx installs things-ENV PATH="$PATH:/root/.local/bin/"diff --git a/docker/update_swe_rex.Dockerfile b/docker/update_swe_rex.Dockerfiledeleted file mode 100644index c8ab9241..00000000--- a/docker/update_swe_rex.Dockerfile+++ /dev/null@@ -1,3 +0,0 @@-FROM sweagent/swe-agent:latest--COPY swe-rex/ /root/swe-rexdiff --git a/docs/config/environments.md b/docs/config/environments.mdnew file mode 100644index 00000000..71623d1b--- /dev/null+++ b/docs/config/environments.md@@ -0,0 +1,33 @@+# Environments++SWE-agent runs on docker images (`python:3.11` by default).+If you are running on SWE-Benmch, every instance has a docker image that we pull from dockerhub.++Here's an example of a simple custom docker environment:++```dockerfile title="tiny.Dockerfile"+FROM python:3.11.10-bullseye # (1)!++ARG DEBIAN_FRONTEND=noninteractive # (2)!+ENV TZ=Etc/UTC++WORKDIR /++# Install swe-rex for faster startup+RUN pip install pipx+RUN pipx install swe-rex+RUN pipx ensurepath+ENV PATH="$PATH:/root/.local/bin/"++# Install any extra dependencies+RUN pip install flake8++SHELL ["/bin/bash", "-c"]+```++1. This is the base image that we're starting from+2. Important to disable any interactive prompts when installing things++Build it with `docker build -f tiny.Dockerfile -t swe-agent-tiny`.++Now you can run it in the agent with `sweagent run --env.image swe-agent-tiny ...`\ No newline at end of filediff --git a/mkdocs.yml b/mkdocs.ymlindex a09a9203..cb69c4a3 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -85,6 +85,7 @@ nav:- "Config files": "config/config.md"- "Templates": "config/templates.md"- "Demonstrations": "config/demonstrations.md"+ - "Environments": "config/environments.md"- Development:- "Contribution guide": "dev/contribute.md"- "Formatting conflicts": "dev/formatting_conflicts.md"