Skip to content

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

NameLast commit date
..
roboarc-blanket-flatten-01-i01-privileged
roboarc-blanket-flatten-01-i01-standard
roboarc-blanket-flatten-02-i01-privileged
roboarc-blanket-flatten-02-i01-standard
roboarc-blanket-flatten-03-i01-privileged
roboarc-blanket-flatten-03-i01-standard
roboarc-blanket-flatten-04-i01-privileged
roboarc-blanket-flatten-04-i01-standard
roboarc-blanket-flatten-05-i01-privileged
roboarc-blanket-flatten-05-i01-standard
roboarc-blanket-flatten-06-i01-privileged
roboarc-blanket-flatten-06-i01-standard
roboarc-blanket-flatten-07-i01-privileged
roboarc-blanket-flatten-07-i01-standard
roboarc-blanket-flatten-08-i01-privileged
roboarc-blanket-flatten-08-i01-standard
roboarc-blanket-fold-01-i01-privileged
roboarc-blanket-fold-01-i01-standard
roboarc-blanket-fold-02-i01-privileged
roboarc-blanket-fold-02-i01-standard
roboarc-blanket-fold-03-i01-privileged
roboarc-blanket-fold-03-i01-standard
roboarc-blanket-fold-04-i01-privileged
roboarc-blanket-fold-04-i01-standard
roboarc-blanket-fold-05-i01-privileged
roboarc-blanket-fold-05-i01-standard
roboarc-buzz-wire-3d-01-i01-privileged
roboarc-buzz-wire-3d-01-i01-standard
roboarc-buzz-wire-3d-02-i01-privileged
roboarc-buzz-wire-3d-02-i01-standard
roboarc-buzz-wire-3d-03-i01-privileged
roboarc-buzz-wire-3d-03-i01-standard
roboarc-buzz-wire-3d-04-i01-privileged
roboarc-buzz-wire-3d-04-i01-standard
roboarc-buzz-wire-3d-05-i01-privileged
roboarc-buzz-wire-3d-05-i01-standard
roboarc-buzz-wire-3d-06-i01-privileged
roboarc-buzz-wire-3d-06-i01-standard
roboarc-buzz-wire-3d-07-i01-privileged
roboarc-buzz-wire-3d-07-i01-standard
roboarc-buzz-wire-3d-08-i01-privileged
roboarc-buzz-wire-3d-08-i01-standard
roboarc-buzz-wire-3d-09-i01-privileged
roboarc-buzz-wire-3d-09-i01-standard
roboarc-buzz-wire-3d-10-i01-privileged
roboarc-buzz-wire-3d-10-i01-standard
roboarc-cast-coil-01-i01-privileged
roboarc-cast-coil-01-i01-standard
roboarc-cast-coil-02-i01-privileged
roboarc-cast-coil-02-i01-standard
roboarc-cast-coil-03-i01-privileged
roboarc-cast-coil-03-i01-standard
roboarc-cast-coil-04-i01-privileged
roboarc-cast-coil-04-i01-standard
roboarc-cast-coil-05-i01-privileged
roboarc-cast-coil-05-i01-standard
roboarc-cast-coil-06-i01-privileged
roboarc-cast-coil-06-i01-standard
roboarc-cast-coil-07-i01-privileged
roboarc-cast-coil-07-i01-standard
roboarc-cast-coil-08-i01-privileged
roboarc-cast-coil-08-i01-standard
roboarc-cast-keyhole-01-i01-privileged
roboarc-cast-keyhole-01-i01-standard
roboarc-cast-keyhole-02-i01-privileged
roboarc-cast-keyhole-02-i01-standard
roboarc-cast-keyhole-03-i01-privileged
roboarc-cast-keyhole-03-i01-standard
roboarc-cast-keyhole-04-i01-privileged
roboarc-cast-keyhole-04-i01-standard
roboarc-cast-keyhole-05-i01-privileged
roboarc-cast-keyhole-05-i01-standard
roboarc-cast-keyhole-06-i01-privileged
roboarc-cast-keyhole-06-i01-standard
roboarc-fiber-bundling-01-i01-privileged
roboarc-fiber-bundling-01-i01-standard
roboarc-fiber-bundling-02-i01-privileged
roboarc-fiber-bundling-02-i01-standard
roboarc-fiber-bundling-03-i01-privileged
roboarc-fiber-bundling-03-i01-standard
roboarc-fiber-bundling-04-i01-privileged
roboarc-fiber-bundling-04-i01-standard
roboarc-fiber-bundling-05-i01-privileged
roboarc-fiber-bundling-05-i01-standard
roboarc-fiber-bundling-06-i01-privileged
roboarc-fiber-bundling-06-i01-standard
roboarc-fiber-bundling-07-i01-privileged
roboarc-fiber-bundling-07-i01-standard
roboarc-fiber-bundling-08-i01-privileged
roboarc-fiber-bundling-08-i01-standard
roboarc-fiber-bundling-09-i01-privileged
roboarc-fiber-bundling-09-i01-standard
roboarc-fiber-bundling-10-i01-privileged
roboarc-fiber-bundling-10-i01-standard
roboarc-hedgehog-in-cage-01-i01-privileged
roboarc-hedgehog-in-cage-01-i01-standard
roboarc-hedgehog-in-cage-02-i01-privileged
roboarc-hedgehog-in-cage-02-i01-standard
roboarc-hedgehog-in-cage-03-i01-privileged
roboarc-hedgehog-in-cage-03-i01-standard
roboarc-hedgehog-in-cage-04-i01-privileged
roboarc-hedgehog-in-cage-04-i01-standard
roboarc-hedgehog-in-cage-05-i01-privileged
roboarc-hedgehog-in-cage-05-i01-standard
roboarc-hedgehog-in-cage-06-i01-privileged
roboarc-hedgehog-in-cage-06-i01-standard
roboarc-hedgehog-in-cage-07-i01-privileged
roboarc-hedgehog-in-cage-07-i01-standard
roboarc-hedgehog-in-cage-08-i01-privileged
roboarc-hedgehog-in-cage-08-i01-standard
roboarc-horseshoe-ring-01-i01-privileged
roboarc-horseshoe-ring-01-i01-standard
roboarc-horseshoe-ring-02-i01-privileged
roboarc-horseshoe-ring-02-i01-standard
roboarc-horseshoe-ring-03-i01-privileged
roboarc-horseshoe-ring-03-i01-standard
roboarc-horseshoe-ring-04-i01-privileged
roboarc-horseshoe-ring-04-i01-standard
roboarc-horseshoe-ring-05-i01-privileged
roboarc-horseshoe-ring-05-i01-standard
roboarc-ikea-eket-i00-privileged
roboarc-ikea-eket-i00-standard
roboarc-ikea-gladom-i00-privileged
roboarc-ikea-gladom-i00-standard
roboarc-ikea-ikea365-i00-privileged
roboarc-ikea-ikea365-i00-standard
roboarc-ikea-lack-i00-privileged
roboarc-ikea-lack-i00-standard
roboarc-ikea-stegesund-i00-privileged
roboarc-ikea-stegesund-i00-standard
roboarc-ikea-svallet-i00-privileged
roboarc-ikea-svallet-i00-standard
roboarc-ikea-variera-i00-privileged
roboarc-ikea-variera-i00-standard
roboarc-ikea-vesken-i00-privileged
roboarc-ikea-vesken-i00-standard
roboarc-interlocked-claws-01-i01-privileged
roboarc-interlocked-claws-01-i01-standard
roboarc-interlocked-claws-02-i01-privileged
roboarc-interlocked-claws-02-i01-standard
roboarc-interlocked-claws-03-i01-privileged
roboarc-interlocked-claws-03-i01-standard
roboarc-interlocked-claws-04-i01-privileged
roboarc-interlocked-claws-04-i01-standard
roboarc-interlocked-claws-05-i01-privileged
roboarc-interlocked-claws-05-i01-standard
roboarc-interlocked-claws-06-i01-privileged
roboarc-interlocked-claws-06-i01-standard
roboarc-interlocked-claws-07-i01-privileged
roboarc-interlocked-claws-07-i01-standard
roboarc-interlocked-claws-08-i01-privileged
roboarc-interlocked-claws-08-i01-standard
roboarc-interlocked-claws-09-i01-privileged
roboarc-interlocked-claws-09-i01-standard
roboarc-interlocked-claws-10-i01-privileged
roboarc-interlocked-claws-10-i01-standard
roboarc-interlocked-claws-11-i01-privileged
roboarc-interlocked-claws-11-i01-standard
roboarc-interlocked-claws-12-i01-privileged
roboarc-interlocked-claws-12-i01-standard
roboarc-knot-untying-01-i01-privileged
roboarc-knot-untying-01-i01-standard
roboarc-knot-untying-02-i01-privileged
roboarc-knot-untying-02-i01-standard
roboarc-knot-untying-03-i01-privileged
roboarc-knot-untying-03-i01-standard
roboarc-knot-untying-04-i01-privileged
roboarc-knot-untying-04-i01-standard
roboarc-knot-untying-05-i01-privileged
roboarc-knot-untying-05-i01-standard
roboarc-knot-untying-06-i01-privileged
roboarc-knot-untying-06-i01-standard
roboarc-knot-untying-09-i01-privileged
roboarc-knot-untying-09-i01-standard
roboarc-lock-picking-01-i01-privileged
roboarc-lock-picking-01-i01-standard
roboarc-lock-picking-02-i01-privileged
roboarc-lock-picking-02-i01-standard
roboarc-lock-picking-03-i01-privileged
roboarc-lock-picking-03-i01-standard
roboarc-lock-picking-04-i01-privileged
roboarc-lock-picking-04-i01-standard
roboarc-lock-picking-05-i01-privileged
roboarc-lock-picking-05-i01-standard
roboarc-orbits-01-i01-privileged
roboarc-orbits-01-i01-standard
roboarc-rope-threading-01-i01-privileged
roboarc-rope-threading-01-i01-standard
roboarc-rope-threading-02-i01-privileged
roboarc-rope-threading-02-i01-standard
roboarc-rope-threading-03-i01-privileged
roboarc-rope-threading-03-i01-standard
roboarc-rope-threading-04-i01-privileged
roboarc-rope-threading-04-i01-standard
roboarc-rope-threading-05-i01-privileged
roboarc-rope-threading-05-i01-standard
roboarc-rope-threading-06-i01-privileged
roboarc-rope-threading-06-i01-standard
roboarc-rope-threading-07-i01-privileged
roboarc-rope-threading-07-i01-standard
roboarc-rope-threading-08-i01-privileged
roboarc-rope-threading-08-i01-standard
roboarc-slotted-post-ring-01-i01-privileged
roboarc-slotted-post-ring-01-i01-standard
roboarc-slotted-post-ring-02-i01-privileged
roboarc-slotted-post-ring-02-i01-standard
roboarc-slotted-post-ring-03-i01-privileged
roboarc-slotted-post-ring-03-i01-standard
roboarc-slotted-post-ring-04-i01-privileged
roboarc-slotted-post-ring-04-i01-standard
roboarc-slotted-post-ring-05-i01-privileged
roboarc-slotted-post-ring-05-i01-standard
README.md