Skip to content

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

NameLast commit date
..
robocasa365-add-lemon-to-fish-i00-privileged
robocasa365-add-lemon-to-fish-i00-standard
robocasa365-assemble-cooking-array-i00-privileged
robocasa365-assemble-cooking-array-i00-standard
robocasa365-beverage-sorting-i00-privileged
robocasa365-beverage-sorting-i00-standard
robocasa365-bread-and-cheese-i00-privileged
robocasa365-bread-and-cheese-i00-standard
robocasa365-candle-cleanup-i00-privileged
robocasa365-candle-cleanup-i00-standard
robocasa365-cereal-and-bowl-i00-privileged
robocasa365-cereal-and-bowl-i00-standard
robocasa365-choose-measuring-cup-i00-privileged
robocasa365-choose-measuring-cup-i00-standard
robocasa365-clear-food-waste-i00-privileged
robocasa365-clear-food-waste-i00-standard
robocasa365-clear-sink-i00-privileged
robocasa365-clear-sink-i00-standard
robocasa365-close-drawer-i00-privileged
robocasa365-close-drawer-i00-standard
robocasa365-close-oven-i00-privileged
robocasa365-close-oven-i00-standard
robocasa365-close-stand-mixer-head-i00-privileged
robocasa365-close-stand-mixer-head-i00-standard
robocasa365-cookie-dough-prep-i00-privileged
robocasa365-cookie-dough-prep-i00-standard
robocasa365-deliver-brewed-coffee-i00-privileged
robocasa365-deliver-brewed-coffee-i00-standard
robocasa365-dessert-upgrade-i00-privileged
robocasa365-dessert-upgrade-i00-standard
robocasa365-display-meat-variety-i00-privileged
robocasa365-display-meat-variety-i00-standard
robocasa365-divide-basins-i00-privileged
robocasa365-divide-basins-i00-standard
robocasa365-drain-veggies-i00-privileged
robocasa365-drain-veggies-i00-standard
robocasa365-drinkware-consolidation-i00-privileged
robocasa365-drinkware-consolidation-i00-standard
robocasa365-make-ice-lemonade-i00-privileged
robocasa365-make-ice-lemonade-i00-standard
robocasa365-make-loaded-potato-i00-privileged
robocasa365-make-loaded-potato-i00-standard
robocasa365-meal-prep-staging-i00-privileged
robocasa365-meal-prep-staging-i00-standard
robocasa365-microwave-defrost-meat-i00-privileged
robocasa365-microwave-defrost-meat-i00-standard
robocasa365-open-dishwasher-i00-privileged
robocasa365-open-dishwasher-i00-standard
robocasa365-organize-coffee-condiments-i00-privileged
robocasa365-organize-coffee-condiments-i00-standard
robocasa365-oven-broil-fish-i00-privileged
robocasa365-oven-broil-fish-i00-standard
robocasa365-pack-dessert-i00-privileged
robocasa365-pack-dessert-i00-standard
robocasa365-pick-place-cabinet-to-counter-i00-privileged
robocasa365-pick-place-cabinet-to-counter-i00-standard
robocasa365-pick-place-sink-to-counter-i00-privileged
robocasa365-pick-place-sink-to-counter-i00-standard
robocasa365-plate-steak-meal-i00-privileged
robocasa365-plate-steak-meal-i00-standard
robocasa365-portion-in-tupperware-i00-privileged
robocasa365-portion-in-tupperware-i00-standard
robocasa365-pre-rinse-station-i00-privileged
robocasa365-pre-rinse-station-i00-standard
robocasa365-prep-for-sanitizing-i00-privileged
robocasa365-prep-for-sanitizing-i00-standard
robocasa365-prep-marinating-meat-i00-privileged
robocasa365-prep-marinating-meat-i00-standard
robocasa365-reheat-meal-i00-privileged
robocasa365-reheat-meal-i00-standard
robocasa365-retrieve-meat-i00-privileged
robocasa365-retrieve-meat-i00-standard
robocasa365-sanitize-prep-cutting-board-i00-privileged
robocasa365-sanitize-prep-cutting-board-i00-standard
robocasa365-scrub-bowl-i00-privileged
robocasa365-scrub-bowl-i00-standard
robocasa365-searing-meat-i00-privileged
robocasa365-searing-meat-i00-standard
robocasa365-seasoning-spice-setup-i00-privileged
robocasa365-seasoning-spice-setup-i00-standard
robocasa365-set-up-cutting-station-i00-privileged
robocasa365-set-up-cutting-station-i00-standard
robocasa365-stocking-breakfast-foods-i00-privileged
robocasa365-stocking-breakfast-foods-i00-standard
robocasa365-sweeten-coffee-i00-privileged
robocasa365-sweeten-coffee-i00-standard
robocasa365-thaw-in-sink-i00-privileged
robocasa365-thaw-in-sink-i00-standard
robocasa365-tong-buffet-setup-i00-privileged
robocasa365-tong-buffet-setup-i00-standard
robocasa365-turn-off-microwave-i00-privileged
robocasa365-turn-off-microwave-i00-standard
robocasa365-turn-on-microwave-i00-privileged
robocasa365-turn-on-microwave-i00-standard
robocasa365-turn-on-sink-faucet-i00-privileged
robocasa365-turn-on-sink-faucet-i00-standard
robocasa365-turn-on-stove-i00-privileged
robocasa365-turn-on-stove-i00-standard
robocasa365-turn-on-toaster-oven-i00-privileged
robocasa365-turn-on-toaster-oven-i00-realtime
robocasa365-turn-on-toaster-oven-i00-standard
README.md