The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
Python ai-agentsbenchmarkcoding-agentsembodied-aievaluationharborllm-agentsrobot-learningroboticssimulation
Commits on main
Merge pull request #44 from JamesKrW/dev/pingyue-hb-hard (+3 more)
24b5687 Pingyue Zhang committed 1 month ago
Merge pull request #55 from JamesKrW/fix/maniguard-lock-pins
d6be3fa Kangrui Wang committed 1 month ago
Merge pull request #37 from JamesKrW/gateway/compact-raw (+1 more)
218aa27 Kangrui Wang committed 1 month ago
Merge pull request #31 from JamesKrW/protocol/images-actions (+5 more)
eb4d79d Kangrui Wang committed 1 month ago
Merge pull request #30 from JamesKrW/dev/pingyue-shared-checks
0d6bb66 Pingyue Zhang committed 1 month ago
Merge pull request #12 from JamesKrW/dev/changhe-metaworldplus (+5 more)
6c0b8b9 Changhe Chen committed 1 month ago
Merge pull request #13 from JamesKrW/codeowners-maintainers (+3 more)
accd6d7 Kangrui Wang committed 1 month ago
Merge pull request #11 from JamesKrW/dev/changhe-taxonomy-sawyer
755cf7a Changhe Chen committed 1 month ago
Merge pull request #9 from JamesKrW/dev/pingyue-humanoidbench (+1 more)
bc74c37 Pingyue Zhang committed 1 month ago
Merge pull request #5 from JamesKrW/dev/kangrui (+1 more)
62d7ad5 Kangrui Wang committed 1 month ago
Merge pull request #1 from JamesKrW/dev/kangrui (+1 more)
5a0c781 Kangrui Wang committed 1 month ago
robot_coding_bench v0.1 scaffold: Harbor benchmark for coding agents on robot simulators (+7 more)
b5fabb9 Kangrui Wang committed 2 months ago