Skip to content

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

NameLast commit date
..
robocasa-arrange-vegetables-i00-privileged
robocasa-arrange-vegetables-i00-standard
robocasa-close-double-door-i00-privileged
robocasa-close-double-door-i00-standard
robocasa-close-drawer-i00-privileged
robocasa-close-drawer-i00-standard
robocasa-close-single-door-i00-privileged
robocasa-close-single-door-i00-standard
robocasa-coffee-press-button-i00-privileged
robocasa-coffee-press-button-i00-standard
robocasa-coffee-serve-mug-i00-privileged
robocasa-coffee-serve-mug-i00-standard
robocasa-coffee-setup-mug-i00-privileged
robocasa-coffee-setup-mug-i00-standard
robocasa-microwave-thawing-i00-privileged
robocasa-microwave-thawing-i00-standard
robocasa-open-double-door-i00-privileged
robocasa-open-double-door-i00-standard
robocasa-open-drawer-i00-privileged
robocasa-open-drawer-i00-standard
robocasa-open-single-door-i00-privileged
robocasa-open-single-door-i00-standard
robocasa-pnp-cab-to-counter-i00-privileged
robocasa-pnp-cab-to-counter-i00-standard
robocasa-pnp-counter-to-cab-i00-privileged
robocasa-pnp-counter-to-cab-i00-standard
robocasa-pnp-counter-to-microwave-i00-privileged
robocasa-pnp-counter-to-microwave-i00-standard
robocasa-pnp-counter-to-sink-i00-privileged
robocasa-pnp-counter-to-sink-i00-standard
robocasa-pnp-counter-to-stove-i00-privileged
robocasa-pnp-counter-to-stove-i00-standard
robocasa-pnp-microwave-to-counter-i00-privileged
robocasa-pnp-microwave-to-counter-i00-standard
robocasa-pnp-sink-to-counter-i00-privileged
robocasa-pnp-sink-to-counter-i00-standard
robocasa-pnp-stove-to-counter-i00-privileged
robocasa-pnp-stove-to-counter-i00-standard
robocasa-pre-soak-pan-i00-privileged
robocasa-pre-soak-pan-i00-standard
robocasa-prepare-coffee-i00-privileged
robocasa-prepare-coffee-i00-standard
robocasa-restock-pantry-i00-privileged
robocasa-restock-pantry-i00-standard
robocasa-turn-off-microwave-i00-privileged
robocasa-turn-off-microwave-i00-standard
robocasa-turn-off-sink-faucet-i00-privileged
robocasa-turn-off-sink-faucet-i00-standard
robocasa-turn-off-stove-i00-privileged
robocasa-turn-off-stove-i00-standard
robocasa-turn-on-microwave-i00-privileged
robocasa-turn-on-microwave-i00-standard
robocasa-turn-on-sink-faucet-i00-privileged
robocasa-turn-on-sink-faucet-i00-standard
robocasa-turn-on-stove-i00-privileged
robocasa-turn-on-stove-i00-standard
robocasa-turn-sink-spout-i00-privileged
robocasa-turn-sink-spout-i00-standard
README.md