Skip to content

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

NameLast commit date
..
humanclaw-nav-chair-i125-privileged
humanclaw-nav-chair-i125-standard
humanclaw-nav-tv-i110-privileged
humanclaw-nav-tv-i110-standard
humanclaw-nav-tv-i111-privileged
humanclaw-nav-tv-i111-standard
humanclaw-nav-tv-i112-privileged
humanclaw-nav-tv-i112-standard
humanclaw-nav-tv-i113-privileged
humanclaw-nav-tv-i113-standard
humanclaw-nav-tv-i114-privileged
humanclaw-nav-tv-i114-standard
humanclaw-nav-tv-i115-privileged
humanclaw-nav-tv-i115-standard
humanclaw-nav-tv-i136-privileged
humanclaw-nav-tv-i136-standard
humanclaw-nav-tv-i139-privileged
humanclaw-nav-tv-i139-standard
humanclaw-nav-tv-i16-privileged
humanclaw-nav-tv-i16-standard
humanclaw-nav-tv-i166-privileged
humanclaw-nav-tv-i166-standard
humanclaw-nav-tv-i17-privileged
humanclaw-nav-tv-i17-standard
humanclaw-sit-bed-i04-privileged
humanclaw-sit-bed-i04-standard
humanclaw-sit-bed-i06-privileged
humanclaw-sit-bed-i06-standard
humanclaw-sit-bed-i08-privileged
humanclaw-sit-bed-i08-standard
humanclaw-sit-bed-i112-privileged
humanclaw-sit-bed-i112-standard
humanclaw-sit-bed-i113-privileged
humanclaw-sit-bed-i113-standard
humanclaw-sit-bed-i114-privileged
humanclaw-sit-bed-i114-standard
humanclaw-sit-bed-i115-privileged
humanclaw-sit-bed-i115-standard
humanclaw-sit-bed-i116-privileged
humanclaw-sit-bed-i116-standard
humanclaw-sit-bed-i16-privileged
humanclaw-sit-bed-i16-standard
humanclaw-sit-bed-i161-privileged
humanclaw-sit-bed-i161-standard
humanclaw-sit-bed-i17-privileged
humanclaw-sit-bed-i17-standard
humanclaw-sit-bed-i204-privileged
humanclaw-sit-bed-i204-standard
humanclaw-sit-bed-i24-privileged
humanclaw-sit-bed-i24-standard
humanclaw-sit-bed-i36-privileged
humanclaw-sit-bed-i36-standard
humanclaw-sit-bed-i42-privileged
humanclaw-sit-bed-i42-standard
humanclaw-sit-bed-i43-privileged
humanclaw-sit-bed-i43-standard
humanclaw-sit-bed-i44-privileged
humanclaw-sit-bed-i44-standard
humanclaw-sit-bed-i45-privileged
humanclaw-sit-bed-i45-standard
humanclaw-sit-bed-i62-privileged
humanclaw-sit-bed-i62-standard
humanclaw-sit-bed-i63-privileged
humanclaw-sit-bed-i63-standard
humanclaw-sit-bed-i72-privileged
humanclaw-sit-bed-i72-standard
humanclaw-sit-bed-i97-privileged
humanclaw-sit-bed-i97-standard
humanclaw-sit-couch-i124-privileged
humanclaw-sit-couch-i124-standard
humanclaw-sit-couch-i14-privileged
humanclaw-sit-couch-i14-standard
humanclaw-sit-couch-i154-privileged
humanclaw-sit-couch-i154-standard
humanclaw-sit-couch-i158-privileged
humanclaw-sit-couch-i158-standard
humanclaw-sit-couch-i164-privileged
humanclaw-sit-couch-i164-standard
humanclaw-sit-couch-i179-privileged
humanclaw-sit-couch-i179-standard
humanclaw-sit-couch-i186-privileged
humanclaw-sit-couch-i186-standard
humanclaw-sit-couch-i199-privileged
humanclaw-sit-couch-i199-standard
humanclaw-sit-couch-i213-privileged
humanclaw-sit-couch-i213-standard
humanclaw-sit-couch-i22-privileged
humanclaw-sit-couch-i22-standard
humanclaw-sit-couch-i92-privileged
humanclaw-sit-couch-i92-standard
humanclaw-sit-couch-i98-privileged
humanclaw-sit-couch-i98-standard
humanclaw-sit-toilet-i04-privileged
humanclaw-sit-toilet-i04-standard
humanclaw-sit-toilet-i112-privileged
humanclaw-sit-toilet-i112-standard
humanclaw-sit-toilet-i113-privileged
humanclaw-sit-toilet-i113-standard
humanclaw-sit-toilet-i132-privileged
humanclaw-sit-toilet-i132-standard
humanclaw-sit-toilet-i14-privileged
humanclaw-sit-toilet-i14-standard
humanclaw-sit-toilet-i15-privileged
humanclaw-sit-toilet-i15-standard
humanclaw-sit-toilet-i66-privileged
humanclaw-sit-toilet-i66-standard
humanclaw-sit-toilet-i91-privileged
humanclaw-sit-toilet-i91-standard
humanclaw-sit-toilet-i95-privileged
humanclaw-sit-toilet-i95-standard
README.md