Skip to content

The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

Shared: Panda Dynamic Scenes README row, taxonomy, label, checks and resource rules (#178)

Changhe Chena58de3ec244e
diff --git a/README.md b/README.md
index 1984391ef5..bbd4bd70f2 100644
--- a/README.md
+++ b/README.md
@@ -66,6 +66,7 @@ drives it from a controller program of its own. It is offered where a benchmark'
| LIBERO-MAX (`libero-max-<suite>-<task>-<event>-iNN`) | robosuite 1.4 (MuJoCo 3.2.6, CPU) · Franka Panda; LIBERO tasks with one change in mid-task (a target, receptacle or obstacle moves, the camera, lighting or scene theme changes, sensor noise sets in, distractors appear) | 64 | full |
| ManiGuard (`maniguard-<family>-iNN`) | BEHAVIOR-1K 3.7.2 (OmniGibson on Isaac Sim 4.5) · Franka Panda with long fin-ray fingers; tabletop tasks with a safety specification checked at every control step | 30 (6 families × 5) | full on 14, none on 16 (human review) |
| MolmoSpaces (`molmospaces-<family>-i00`) | Ai2 MolmoSpaces on MuJoCo 3.5 (CPU physics) · Franka FR3 + Robotiq 2F-85 on a fixed base; Rainbow RB-Y1 mobile manipulator | 8 | none (human review) |
+| Panda Dynamic Scenes (`pandadyn-<code>-<name>-i00`) | ManiSkill 3 / SAPIEN (CPU PhysX at 1000 Hz) · one or two Franka Panda arms with task tools (a plate, a cup, a paddle, a scoop, ...); multi-stage dynamic tasks: catch, strike, carry, hand over, release objects that move by themselves | 25 (12 one-arm + 13 two-arm) | full (the benchmark's recorded reference rollouts) |
Each task directory carries a README in one layout: a key table (benchmark, mode, twin, simulator and robot, family and
instance, oracle, budget, image), then Task, Difficulty, Reference solution, Verification and Details.
@@ -80,7 +81,7 @@ instance, oracle, budget, image), then Task, Difficulty, Reference solution, Ver
- **DexToolBench generators:** `scripts/dtb_mk_task.py`, `dtb_mk_oracle.py`.
- **MuJoCo standard-mode twins:** generated by `scripts/mj_mk_limited.py`.
- **KinDER, HumanoidBench:** both modes (HumanoidBench: also standard-realtime) rendered by `scripts/kinder/gen_tasks.py` and `scripts/humanoidbench/gen_tasks.py` (`render`, `check`).
-- **Other benchmarks:** each has its generator under `scripts/<bench>/` (RoboWits, RoboLab, BEHAVIOR-1K, RoboTangle, YAMLab, RoboDojo, ManiGuard, MolmoSpaces: `mk_tasks.py --check`; VLABench, RoboCasa, RoboCasa365, RoboCasa-GR1, LIBERO-MAX, DexJoCo, MuJoCo Playground's quadrupeds: `generate.py --check`; BiGym 2.0: `gen_tasks.py check`; MyoSuite: `scripts/myosuite/generate.py --check`; LabUtopia: `scripts/labutopia/generate.py --check`; AutoBio: `scripts/autobio/generate.py --check`; MetaWorld+: `generate.py`; RoboPaint: `scripts/robopaint_strict/mk_tasks.py --check`, which runs `scripts/robopaint_brush_mk_task.py --refresh` and adds the process rules). CI runs every check mode (`ci_checks/check-generated-tasks.sh`). Per-benchmark notes: [docs/benchmarks/](docs/benchmarks/).
+- **Other benchmarks:** each has its generator under `scripts/<bench>/` (RoboWits, RoboLab, BEHAVIOR-1K, RoboTangle, YAMLab, RoboDojo, ManiGuard, MolmoSpaces, Panda Dynamic Scenes: `mk_tasks.py --check`; VLABench, RoboCasa, RoboCasa365, RoboCasa-GR1, LIBERO-MAX, DexJoCo, MuJoCo Playground's quadrupeds: `generate.py --check`; BiGym 2.0: `gen_tasks.py check`; MyoSuite: `scripts/myosuite/generate.py --check`; LabUtopia: `scripts/labutopia/generate.py --check`; AutoBio: `scripts/autobio/generate.py --check`; MetaWorld+: `generate.py`; RoboPaint: `scripts/robopaint_strict/mk_tasks.py --check`, which runs `scripts/robopaint_brush_mk_task.py --refresh` and adds the process rules). CI runs every check mode (`ci_checks/check-generated-tasks.sh`). Per-benchmark notes: [docs/benchmarks/](docs/benchmarks/).
- **RoboFunOlympics:** `scripts/robofunolympics/gen_tasks.py` (`render`, `check`) from the families under
`scripts/robofunolympics/shared/rfo/tasks/`; how to write a family: `scripts/robofunolympics/AUTHORING.md`; notes in
[docs/benchmarks/robofunolympics.md](docs/benchmarks/robofunolympics.md).
diff --git a/ci_checks/check-generated-tasks.sh b/ci_checks/check-generated-tasks.sh
index 505ab5b32a..907c2f89ac 100755
--- a/ci_checks/check-generated-tasks.sh
+++ b/ci_checks/check-generated-tasks.sh
@@ -39,6 +39,10 @@ if [ -f "$root/scripts/yamlab/mk_tasks.py" ]; then # YAMLab (its tasks pull re
if out=$(python3 "$root/scripts/yamlab/mk_tasks.py" --check 2>&1); then echo "ok scripts/yamlab/mk_tasks.py: ${out##*$'\n'}"
else echo "FAIL scripts/yamlab/mk_tasks.py --check:"; echo "$out" | tail -n 40; rc=1; fi
fi
+if [ -f "$root/scripts/pandadyn/mk_tasks.py" ]; then # Panda Dynamic Scenes (its tasks pull request adds the generator)
+ if out=$(python3 "$root/scripts/pandadyn/mk_tasks.py" --check 2>&1); then echo "ok scripts/pandadyn/mk_tasks.py: ${out##*$'\n'}"
+ else echo "FAIL scripts/pandadyn/mk_tasks.py --check:"; echo "$out" | tail -n 40; rc=1; fi
+fi
if [ -f "$root/scripts/robodojo/mk_tasks.py" ]; then # RoboDojo (its tasks pull request adds the generator)
if out=$(python3 "$root/scripts/robodojo/mk_tasks.py" --check 2>&1); then echo "ok scripts/robodojo/mk_tasks.py: ${out##*$'\n'}"
else echo "FAIL scripts/robodojo/mk_tasks.py --check:"; echo "$out" | tail -n 40; rc=1; fi
diff --git a/ci_checks/check-tooling-identity.sh b/ci_checks/check-tooling-identity.sh
index 0c32156157..aa580d7095 100755
--- a/ci_checks/check-tooling-identity.sh
+++ b/ci_checks/check-tooling-identity.sh
@@ -1,13 +1,13 @@
#!/usr/bin/env bash
# Shared-source rules (PROTOCOL §2.1, D9: copies of shared sources are byte-identical). Two parts:
-# 1. BEHAVIOR-1K, RoboLab, RoboWits, ManiGuard, MolmoSpaces, MyoSuite, DexJoCo, RoboTangle, BiGym 2.0, YAMLab, RoboDojo, LabUtopia and AutoBio: within one benchmark, every mode, every copy of a shared file is byte-identical
+# 1. BEHAVIOR-1K, RoboLab, RoboWits, ManiGuard, MolmoSpaces, MyoSuite, DexJoCo, RoboTangle, BiGym 2.0, YAMLab, RoboDojo, LabUtopia, AutoBio and Panda Dynamic Scenes: within one benchmark, every mode, every copy of a shared file is byte-identical
# (the generators copy them from the benchmark's template task unchanged):
# - the tooling module rcb_<sim>.py wherever it sits (privileged environment/ and tests/, standard environment/sim/);
# - the standard agent side environment/rcb_client.py + rcb1.py and the sim service environment/sim/ (server.py,
# finalize.sh, run.sh, rcb1.py);
# - the graders, tests/grade.py (one version per mode).
# A change to one of them is rolled out to every task of the benchmark at once. Same approach as check-robopaint-tooling.sh.
-# 2. The eai-standard/2.2 layer (every task whose metadata.protocol is 2.2: BEHAVIOR, RoboLab, RoboWits, RoboPaint, MolmoSpaces, MyoSuite, DexJoCo, YAMLab, RoboDojo, LabUtopia, AutoBio): the
+# 2. The eai-standard/2.2 layer (every task whose metadata.protocol is 2.2: BEHAVIOR, RoboLab, RoboWits, RoboPaint, MolmoSpaces, MyoSuite, DexJoCo, YAMLab, RoboDojo, LabUtopia, AutoBio, Panda Dynamic Scenes): the
# agent's client environment/rcb_client.py + rcb1.py and the service's environment/sim/rcb_limited.py + rcb1.py +
# run.sh are byte-identical to their source in scripts/eai_standard/ (docs/STANDARD_MODE.md), and the sidecar is
# set up for crash recovery: `command: sh /srv/run.sh` (run.sh exits non-zero when the service crashes) and
@@ -39,7 +39,8 @@ tasks, src = sys.argv[1], sys.argv[2]
MODULE = {"behavior": "rcb_behavior.py", "robolab": "rcb_robolab.py", "robowits": "rcb_robowits.py", "maniguard": "rcb_maniguard.py",
"molmospaces": "rcb_molmospaces.py", "myosuite": "rcb_myosuite.py",
"dexjoco": "rcb_dexjoco.py", "robotangle": "rcb_topogym.py", "bigym2": "rcb_bigym2.py", "yamlab": "rcb_yamlab.py", "robodojo": "rcb_robodojo.py",
- "labutopia": "rcb_labutopia.py", "autobio": "rcb_autobio.py", "robofunolympics": "rcb_rfo.py"}
+ "labutopia": "rcb_labutopia.py", "autobio": "rcb_autobio.py", "robofunolympics": "rcb_rfo.py",
+ "pandadyn": "rcb_pandadyn.py"}
SHARED = {"environment/rcb_client.py", "environment/rcb1.py", "environment/sim/server.py", "environment/sim/finalize.sh",
"environment/sim/run.sh", "environment/sim/rcb1.py", "environment/sim/rcb_limited.py", "tests/grade.py"}
LAYER = {"environment/rcb_client.py": "rcb_client.py", "environment/rcb1.py": "rcb1.py", "environment/sim/rcb_limited.py": "rcb_limited.py",
diff --git a/ci_checks/check-unit-tests.sh b/ci_checks/check-unit-tests.sh
index 0fd122f177..539a119fe5 100755
--- a/ci_checks/check-unit-tests.sh
+++ b/ci_checks/check-unit-tests.sh
@@ -47,6 +47,8 @@ if python3 -c "import numpy, requests" 2>/dev/null; then
robodojo/tests/test_privileged_grading.py; do
run 900 "$t"
done
+ # Panda Dynamic Scenes' graders, privileged and standard (its tasks pull request adds the test)
+ if [ -f "$root/scripts/pandadyn/tests/test_grading.py" ]; then run 300 pandadyn/tests/test_grading.py; fi
# LIBERO-MAX's rcb_service.py is its own variant (its verdict and video protocol differ): the classes of the shared
# service test that apply to it, run against that copy
RCB_SERVICE_DIR="$root/scripts/libero-max/shared/standard" run 900 humanoidbench/tests/test_standard_service.py \
diff --git a/docs/PROTOCOL.md b/docs/PROTOCOL.md
index e7363ed239..ae99510763 100644
--- a/docs/PROTOCOL.md
+++ b/docs/PROTOCOL.md
@@ -491,8 +491,8 @@ under an unrelated name would be missed.
| LIBERO-MAX | `libero`, `hf-libero`, `rlinf-libero`, `rpent-libero`, `praxis-libero`, `libero-infinity`, `rlinf-liberopro`, `rpent-liberopro`, `rlinf-liberoplus` | LIBERO's goal definitions and initial states; no meshes, no demonstrations, not LIBERO-MAX's events |
Not on PyPI under any name found: RoboTwin's own package, DexToolBench / SimToolReal, HumanoidBench, RoboWits, RoboLab,
-ManiGuard, DexJoCo, LabUtopia, AutoBio, VLABench, RoboDojo, RoboTangle, RoboOlympics, RoboFunOlympics, BiGym, YAMLab; Isaac Sim does not
-install from pypi.org.
+ManiGuard, DexJoCo, LabUtopia, AutoBio, VLABench, RoboDojo, RoboTangle, RoboOlympics, RoboFunOlympics, BiGym, YAMLab, Panda Dynamic Scenes
+(not published at all: its environment package is this repository's `images/pandadyn/`); Isaac Sim does not install from pypi.org.
**Per-benchmark exceptions.** A benchmark's agents have open egress, and its instructions say so, only by owner
decision, recorded here, in `docs/benchmarks/<bench>.md` and in the three lists that implement it (the same directory
@@ -657,7 +657,7 @@ Wire details (endpoints, RCB1 frames, `meta`, error codes, crash recovery) are i
| Version | Date | Content |
|---|---|---|
| eai-standard/2.1 | 2026-09-30 | the base (formerly `rcb-limited/2.1`): no reset; `/submit` action chunks, `/observe`; binary RCB1 responses; 82-column proprioception, optional depth noise; crash recovery. Served by the standard tasks of RoboTwin, DexToolBench, MuJoCo Playground (manipulation), KinDER, HumanoidBench, MetaWorld+, VLABench, the three RoboCasa benchmarks, LIBERO-MAX, BiGym 2.0 and RoboFunOlympics |
-| eai-standard/2.2 | 2026-10-01 | additive extension (§6, [`STANDARD_MODE.md`](STANDARD_MODE.md) Part 2) with crash recovery; served by BEHAVIOR-1K, RoboLab, RoboWits, RoboPaint, MolmoSpaces, MyoSuite, DexJoCo, YAMLab, RoboDojo, LabUtopia, AutoBio, ManiGuard and MuJoCo Playground's quadruped tasks |
+| eai-standard/2.2 | 2026-10-01 | additive extension (§6, [`STANDARD_MODE.md`](STANDARD_MODE.md) Part 2) with crash recovery; served by BEHAVIOR-1K, RoboLab, RoboWits, RoboPaint, MolmoSpaces, MyoSuite, DexJoCo, YAMLab, RoboDojo, LabUtopia, AutoBio, ManiGuard, Panda Dynamic Scenes and MuJoCo Playground's quadruped tasks |
| eai-standard/2.3 | 2026-10-08 | 2.2 + the realtime capability (§4b; [`STANDARD_MODE.md`](STANDARD_MODE.md) Part 3, "Standard-realtime mode"); every 2.2 addition stays a MAY. Served by standard-realtime tasks only (HumanoidBench, RoboOlympics, BiGym 2.0); standard tasks stay on 2.1 / 2.2. Issue #86 |
| eai-standard/2.4 | 2026-10-08 | 2.3 + the operations capability ([`STANDARD_MODE.md`](STANDARD_MODE.md) Part 4): operation ids and request deadlines (408 / 409), `GET /operations/<id>`, a service-owned call watchdog, a physics-tick ledger, fsynced journals; every earlier addition, the realtime capability included, stays a MAY. Served by standard tasks (RoboTangle); reference implementation `scripts/eai_standard/v2_4/`. Issue #89 |
diff --git a/docs/STANDARD_MODE.md b/docs/STANDARD_MODE.md
index c2b81d8afa..d51619d767 100644
--- a/docs/STANDARD_MODE.md
+++ b/docs/STANDARD_MODE.md
@@ -227,10 +227,11 @@ episode is graded. Results with `N > 0` are reported separately and never ranked
> says otherwise, every 2.2 addition is a MAY capability, and a request written for 2.0 or 2.1 gets the same answer as
> before (plus parts and keys an old client ignores). Which parts are required is listed in PROTOCOL §4.
-**Who serves it.** The standard twins of twelve benchmarks: BEHAVIOR-1K, RoboLab, RoboWits, RoboPaint, ManiGuard, MolmoSpaces,
-MyoSuite, DexJoCo, YAMLab, RoboDojo, LabUtopia and AutoBio ([docs/benchmarks/](benchmarks/); MyoSuite and AutoBio: MuJoCo on the CPU, resume class `exact`; YAMLab: Isaac
+**Who serves it.** The standard twins of thirteen benchmarks: BEHAVIOR-1K, RoboLab, RoboWits, RoboPaint, ManiGuard, MolmoSpaces,
+MyoSuite, DexJoCo, YAMLab, RoboDojo, LabUtopia, AutoBio and Panda Dynamic Scenes ([docs/benchmarks/](benchmarks/); MyoSuite and AutoBio: MuJoCo on the CPU, resume class `exact`; YAMLab: Isaac
Lab with CPU PhysX, resume class `exact`; RoboDojo: Isaac Lab with GPU PhysX, resume class `exact`, `approx` for its six
-standard-only tasks graded by the live episode; LabUtopia: Isaac Sim, PhysX on the CPU, resume class `exact`), and MuJoCo Playground's quadruped tasks. The RoboTwin and the other MuJoCo standard tasks (DexToolBench, MuJoCo Playground's manipulation tasks) serve 2.1.
+standard-only tasks graded by the live episode; LabUtopia: Isaac Sim, PhysX on the CPU, resume class `exact`; Panda Dynamic Scenes:
+ManiSkill / SAPIEN with CPU PhysX, resume class `exact`), and MuJoCo Playground's quadruped tasks. The RoboTwin and the other MuJoCo standard tasks (DexToolBench, MuJoCo Playground's manipulation tasks) serve 2.1.
**One implementation.** The source is [`scripts/eai_standard/`](../scripts/eai_standard/), the D9 shared source of these six
benchmarks: the service layer `rcb_limited.py` (HTTP, RCB1 responses, the protocol helpers, the `/spec` self-check, crash
@@ -443,7 +444,7 @@ The resume check depends on the simulator's determinism class, which the adapter
| class | rule | benchmarks |
|---|---|---|
-| `exact` | the replayed state hash must equal the checkpoint's; otherwise the episode ends `resume_failed` (`infra_error = 1`) | RoboWits on Genesis' CPU backend; RoboLab (Isaac, CPU PhysX: the resumed service is a fresh process, whose first replay is bit-exact); RoboPaint (ManiSkill/SAPIEN, CPU PhysX, CPU brush model); MolmoSpaces (MuJoCo, CPU physics); YAMLab (Isaac, CPU PhysX, a fixed env seed: the resumed service is a fresh process, whose first run is bit-exact); LabUtopia (Isaac Sim, CPU PhysX, a fresh process as RoboLab; one control step is one physics step); AutoBio (MuJoCo 3.3, CPU physics; one control step is one physics step); RoboDojo (Isaac, GPU PhysX: the resumed service is a fresh process, whose first run is bit-exact, measured on 48 of its 54 tasks; `make_toast` with PhysX enhanced determinism) |
+| `exact` | the replayed state hash must equal the checkpoint's; otherwise the episode ends `resume_failed` (`infra_error = 1`) | RoboWits on Genesis' CPU backend; RoboLab (Isaac, CPU PhysX: the resumed service is a fresh process, whose first replay is bit-exact); RoboPaint (ManiSkill/SAPIEN, CPU PhysX, CPU brush model); MolmoSpaces (MuJoCo, CPU physics); YAMLab (Isaac, CPU PhysX, a fixed env seed: the resumed service is a fresh process, whose first run is bit-exact); LabUtopia (Isaac Sim, CPU PhysX, a fresh process as RoboLab; one control step is one physics step); AutoBio (MuJoCo 3.3, CPU physics; one control step is one physics step); RoboDojo (Isaac, GPU PhysX: the resumed service is a fresh process, whose first run is bit-exact, measured on 48 of its 54 tasks; `make_toast` with PhysX enhanced determinism); Panda Dynamic Scenes (ManiSkill / SAPIEN, CPU PhysX with enhanced determinism, one thread; every reset rebuilds the scene; ten physics steps per control step) |
| `approx` | the hash is compared and the result recorded; a mismatch does not end the episode: `resume_state_mismatch = 1`, and the robot's deviation from its checkpointed proprio is recorded (max \|Δ\| over `joint_pos`; the largest TCP position error from `tcp_pose`). Only a gross divergence, a TCP more than `RCB_RESUME_MAX_DEV_M` (0.05 m) from where it was, ends the episode as `resume_failed` (`infra_error = 1`) | simulators declared nondeterministic: RoboWits on the GPU backend (the particle families separate_marbles_and_sand and stabilize_bottle, `RCB_ROBOWITS_BACKEND=gpu`); BEHAVIOR-1K (OmniGibson declares rollouts nondeterministic; measured so far: live = fresh replay on every 2.2 run, so it normally resumes exactly); ManiGuard (OmniGibson 3.7.2; not measured yet); RoboDojo's six standard-only tasks graded by the live episode (`make_toast_random`, the three liquid tasks and the two cloth tasks: GPU PhysX particles and cloth do not replay bit for bit; PROTOCOL §4.8) |
This relaxes "the hash checked" of PROTOCOL §4.7 for declared-nondeterministic simulators only. It cannot raise a score:
diff --git a/docs/TAXONOMY.md b/docs/TAXONOMY.md
index ebeb86dfd3..1b954473b6 100644
--- a/docs/TAXONOMY.md
+++ b/docs/TAXONOMY.md
@@ -7,9 +7,9 @@ Known values; a new value is added by pull request. See [PROTOCOL.md](PROTOCOL.m
| `author_name` | the author's name (an organization may go in `author_org`) |
| `task_type` | `trajectory` · `policy` |
| `mode` | `privileged` · `standard` (formerly `unlimited` / `limited`) · `standard-realtime` (protocol v1.1; named so since v1.1.1, PROTOCOL §4b) |
-| `protocol` | standard and standard-realtime mode only. Standard-realtime: `eai-standard/2.3`. Standard: `eai-standard/2.1` · `eai-standard/2.2` (BEHAVIOR-1K, RoboLab, RoboWits, RoboPaint, ManiGuard, MolmoSpaces, MyoSuite, DexJoCo, MuJoCo Playground's quadruped tasks, YAMLab, RoboDojo, LabUtopia, AutoBio) · `eai-standard/2.4` (the operations capability: RoboTangle). All are specified in [STANDARD_MODE.md](STANDARD_MODE.md) |
-| `simulator` | `robosuite` · `robotwin` (RoboTwin 2.0 = SAPIEN 3 + CuRobo) · `mujoco` · `kinder` (KinDER: MuJoCo and PyBullet) · `humanoidbench` (HumanoidBench: MuJoCo + dm_control) · `robowits` (RoboWits on Genesis) · `robolab` (NVIDIA RoboLab on Isaac Lab) · `labutopia` (LabUtopia on Isaac Sim) · `behavior` (BEHAVIOR-1K: OmniGibson on Isaac Sim) · `maniskill` (ManiSkill 3 / SAPIEN: RoboPaint) · `vlabench` (VLABench: MuJoCo + dm_control) · `maniguard` (ManiGuard: BEHAVIOR-1K 3.7.2, OmniGibson on Isaac Sim 4.5) · `molmospaces` (Ai2 MolmoSpaces: MuJoCo) · `myosuite` (MyoSuite: musculoskeletal models on MuJoCo) · `dexjoco` (DexJoCo: MuJoCo 3.4 + its gymnasium envs) · `bigym2` (BiGym 2.0: MuJoCo 3.8.1 with NVIDIA's GR00T-WBC lower-body policy in every step) · `robotangle` (RoboTangle, formerly TopoGym: Genesis 1.2.1 rigid bodies and libuipc IPC / PBD deformables, Nyx RGB) · `yamlab` (YAMLab on Isaac Lab / Isaac Sim 5.1, PhysX on the CPU) · `robodojo` (RoboDojo's simulation benchmark: its Isaac Lab 2.3 fork on Isaac Sim 5.1, PhysX with GPU dynamics, cuRobo IK) · `autobio` (AutoBio: MuJoCo 3.3 with AutoBio's own thread, detent and liquid-surface mechanisms) — the simulator, not the image: benchmarks on one simulator may use different images and versions (PROTOCOL D5) |
-| `robot` | `franka-panda` · `aloha-agilex` · `aloha-2` (ALOHA 2, MuJoCo Menagerie: MuJoCo Playground, RoboTangle) · `kuka-iiwa14-sharpa-ha4` · `tidybot-plus-plus` · `unitree-h1` · `unitree-h1hand` (H1 with two Shadow hands) · `sawyer` (Rethink Robotics Sawyer, as in Meta-World) · `marvin-bimanual` (RoboWits) · `franka-robotiq-2f85` (a Franka arm with a Robotiq 2F-85, the DROID setup: RoboLab, MolmoSpaces' FR3, RoboTangle's Panda) · `r1pro` (BEHAVIOR-1K) · `franka-panda-omron` (RoboCasa's PandaOmron: Franka Panda on an Omron mobile base with a lift torso) · `fourier-gr1` (Fourier GR1 humanoid: two arms, waist and two Fourier dexterous hands, fixed base; RoboCasa-GR1) · `franka-panda-longfinger` (ManiGuard: Franka Panda with long fin-ray fingers) · `rainbow-rby1` (Rainbow Robotics RB-Y1: holonomic base, 6-joint torso, two 7-joint arms with parallel grippers; MolmoSpaces) · `myohand` (MyoSuite's MyoHand: a musculoskeletal human forearm and hand, 39 muscles) · `myoarm` (MyoSuite's MyoArm: a musculoskeletal human shoulder, arm and hand, 34 or 63 muscles) · `myoarm-torso` (MyoSuite's table-tennis player: torso, right arm and hand on a sliding pelvis) · `myoarm-mpl` (MyoSuite's MyoArm next to the Modular Prosthetic Limb, a motorized arm) · `myolegs` (MyoSuite's MyoLegs: a free-standing musculoskeletal torso and both legs) · `myoleg-osl` (MyoSuite's MyoLeg with an Open-Source Leg knee-ankle prosthesis) · `unitree-go1` (Unitree Go1 quadruped; MuJoCo Playground) · `boston-dynamics-spot` (Boston Dynamics Spot quadruped; MuJoCo Playground) · `google-barkour-vb` (Google Barkour vB quadruped; MuJoCo Playground) · `franka-allegro` (Franka Panda with a 16-DoF Allegro hand; DexJoCo) · `franka-allegro-bimanual` (two of them side by side; DexJoCo) · `kuka-iiwa14` (KUKA LBR iiwa 14 with a tool, here an air hockey mallet; RoboOlympics) · `unitree-g1` (Unitree G1 29-DoF humanoid with its rubber hands, a switchable grip on each palm; RoboOlympics' robot library with Unitree's limits and motor curves; RoboFunOlympics) · `unitree-h1-2` (Unitree H1-2, the adult-size 27-DoF humanoid, with the same grip; RoboOlympics' robot library; RoboFunOlympics' tasks whose official venue needs an adult-size body) · `unitree-g1-dex1` (Unitree G1 29-DoF humanoid with Dex1-1 two-finger grippers; legs and waist driven by GR00T-WBC; BiGym 2.0) · `i2rt-yam-bimanual` (two I2RT YAM arms, 6 joints each, with parallel two-finger grippers on a workstation table; YAMLab) · `arx-x5-bimanual` (two ARX X5 arms, 6 joints each, with parallel two-finger grippers at a table; RoboDojo) · `aloha-2-arm` (one arm of ALOHA 2 with its parallel gripper, fixed to a bench; AutoBio, whose two-arm tasks use `aloha-2`) · `ur5e-robotiq-2f85` (a UR5e with a Robotiq 2F-85; AutoBio) · `ur5e-dexhand-2f85-bimanual` (two UR5e arms, one with a DexHand021 dexterous hand that holds a pipette, one with a Robotiq 2F-85; AutoBio) |
+| `protocol` | standard and standard-realtime mode only. Standard-realtime: `eai-standard/2.3`. Standard: `eai-standard/2.1` · `eai-standard/2.2` (BEHAVIOR-1K, RoboLab, RoboWits, RoboPaint, ManiGuard, MolmoSpaces, MyoSuite, DexJoCo, MuJoCo Playground's quadruped tasks, YAMLab, RoboDojo, LabUtopia, AutoBio, Panda Dynamic Scenes) · `eai-standard/2.4` (the operations capability: RoboTangle). All are specified in [STANDARD_MODE.md](STANDARD_MODE.md) |
+| `simulator` | `robosuite` · `robotwin` (RoboTwin 2.0 = SAPIEN 3 + CuRobo) · `mujoco` · `kinder` (KinDER: MuJoCo and PyBullet) · `humanoidbench` (HumanoidBench: MuJoCo + dm_control) · `robowits` (RoboWits on Genesis) · `robolab` (NVIDIA RoboLab on Isaac Lab) · `labutopia` (LabUtopia on Isaac Sim) · `behavior` (BEHAVIOR-1K: OmniGibson on Isaac Sim) · `maniskill` (ManiSkill 3 / SAPIEN: RoboPaint; Panda Dynamic Scenes, ManiSkill 3.0.1 / SAPIEN 3.0.3 with CPU PhysX at 1000 Hz) · `vlabench` (VLABench: MuJoCo + dm_control) · `maniguard` (ManiGuard: BEHAVIOR-1K 3.7.2, OmniGibson on Isaac Sim 4.5) · `molmospaces` (Ai2 MolmoSpaces: MuJoCo) · `myosuite` (MyoSuite: musculoskeletal models on MuJoCo) · `dexjoco` (DexJoCo: MuJoCo 3.4 + its gymnasium envs) · `bigym2` (BiGym 2.0: MuJoCo 3.8.1 with NVIDIA's GR00T-WBC lower-body policy in every step) · `robotangle` (RoboTangle, formerly TopoGym: Genesis 1.2.1 rigid bodies and libuipc IPC / PBD deformables, Nyx RGB) · `yamlab` (YAMLab on Isaac Lab / Isaac Sim 5.1, PhysX on the CPU) · `robodojo` (RoboDojo's simulation benchmark: its Isaac Lab 2.3 fork on Isaac Sim 5.1, PhysX with GPU dynamics, cuRobo IK) · `autobio` (AutoBio: MuJoCo 3.3 with AutoBio's own thread, detent and liquid-surface mechanisms) — the simulator, not the image: benchmarks on one simulator may use different images and versions (PROTOCOL D5) |
+| `robot` | `franka-panda` · `aloha-agilex` · `aloha-2` (ALOHA 2, MuJoCo Menagerie: MuJoCo Playground, RoboTangle) · `kuka-iiwa14-sharpa-ha4` · `tidybot-plus-plus` · `unitree-h1` · `unitree-h1hand` (H1 with two Shadow hands) · `sawyer` (Rethink Robotics Sawyer, as in Meta-World) · `marvin-bimanual` (RoboWits) · `franka-robotiq-2f85` (a Franka arm with a Robotiq 2F-85, the DROID setup: RoboLab, MolmoSpaces' FR3, RoboTangle's Panda) · `r1pro` (BEHAVIOR-1K) · `franka-panda-omron` (RoboCasa's PandaOmron: Franka Panda on an Omron mobile base with a lift torso) · `fourier-gr1` (Fourier GR1 humanoid: two arms, waist and two Fourier dexterous hands, fixed base; RoboCasa-GR1) · `franka-panda-longfinger` (ManiGuard: Franka Panda with long fin-ray fingers) · `rainbow-rby1` (Rainbow Robotics RB-Y1: holonomic base, 6-joint torso, two 7-joint arms with parallel grippers; MolmoSpaces) · `myohand` (MyoSuite's MyoHand: a musculoskeletal human forearm and hand, 39 muscles) · `myoarm` (MyoSuite's MyoArm: a musculoskeletal human shoulder, arm and hand, 34 or 63 muscles) · `myoarm-torso` (MyoSuite's table-tennis player: torso, right arm and hand on a sliding pelvis) · `myoarm-mpl` (MyoSuite's MyoArm next to the Modular Prosthetic Limb, a motorized arm) · `myolegs` (MyoSuite's MyoLegs: a free-standing musculoskeletal torso and both legs) · `myoleg-osl` (MyoSuite's MyoLeg with an Open-Source Leg knee-ankle prosthesis) · `unitree-go1` (Unitree Go1 quadruped; MuJoCo Playground) · `boston-dynamics-spot` (Boston Dynamics Spot quadruped; MuJoCo Playground) · `google-barkour-vb` (Google Barkour vB quadruped; MuJoCo Playground) · `franka-allegro` (Franka Panda with a 16-DoF Allegro hand; DexJoCo) · `franka-allegro-bimanual` (two of them side by side; DexJoCo) · `franka-panda-bimanual` (two stock Franka Panda (FER) arms with their standard hands, facing each other or side by side at one table, each with a rigid task tool on its hand where the task has one; Panda Dynamic Scenes) · `kuka-iiwa14` (KUKA LBR iiwa 14 with a tool, here an air hockey mallet; RoboOlympics) · `unitree-g1` (Unitree G1 29-DoF humanoid with its rubber hands, a switchable grip on each palm; RoboOlympics' robot library with Unitree's limits and motor curves; RoboFunOlympics) · `unitree-h1-2` (Unitree H1-2, the adult-size 27-DoF humanoid, with the same grip; RoboOlympics' robot library; RoboFunOlympics' tasks whose official venue needs an adult-size body) · `unitree-g1-dex1` (Unitree G1 29-DoF humanoid with Dex1-1 two-finger grippers; legs and waist driven by GR00T-WBC; BiGym 2.0) · `i2rt-yam-bimanual` (two I2RT YAM arms, 6 joints each, with parallel two-finger grippers on a workstation table; YAMLab) · `arx-x5-bimanual` (two ARX X5 arms, 6 joints each, with parallel two-finger grippers at a table; RoboDojo) · `aloha-2-arm` (one arm of ALOHA 2 with its parallel gripper, fixed to a bench; AutoBio, whose two-arm tasks use `aloha-2`) · `ur5e-robotiq-2f85` (a UR5e with a Robotiq 2F-85; AutoBio) · `ur5e-dexhand-2f85-bimanual` (two UR5e arms, one with a DexHand021 dexterous hand that holds a pipette, one with a Robotiq 2F-85; AutoBio) |
| `family` | free-form slug naming the scene/goal, shared by its instances (e.g. `stack_blocks_two`, `dextoolbench/brush/blue_brush/sweep_forward`) |
| `oracle` | `full` (oracle scores 1) · `partial` (oracle reaches part of the goal) · `none` — see PROTOCOL §2.7 |
| `oracle_evidence` | required unless `oracle = "full"`: where the evidence for solvability is (PR, results site, upstream record) |
diff --git a/scripts/task_conventions.py b/scripts/task_conventions.py
index dbf65e2def..e0d42efff1 100644
--- a/scripts/task_conventions.py
+++ b/scripts/task_conventions.py
@@ -73,6 +73,7 @@ BENCHMARKS = [ # (directory prefix, label)
("yamlab-", "YAMLab"),
("robodojo-", "RoboDojo"),
("robofunolympics-", "RoboFunOlympics"),
+ ("pandadyn-", "Panda Dynamic Scenes"),
]
# Documented exceptions: tasks without a privileged twin (the same lists: ci_checks/check-instance-copies.sh NO_TWIN and
# STANDARD_ONLY). RoboOlympics (PROTOCOL §4b, owner 2026-10-08): standard-realtime only. RoboDojo's six tasks whose physics
diff --git a/scripts/task_resources.toml b/scripts/task_resources.toml
index cf6aa365f6..7434348282 100644
--- a/scripts/task_resources.toml
+++ b/scripts/task_resources.toml
@@ -820,6 +820,34 @@ verifier_min_memory_gb = 3.0
verifier_min_gpu_memory_gb = 0.0
measured = "author estimate (ws-1, 2026-10-11): CPU-only; the sim sidecar steps and renders on one or two cores (under 1 GB), the agent's container holds the harness and the client"
+# pandadyn, privileged: 25 task(s)
+[[rule]]
+match = "^pandadyn-.*-privileged$"
+min_cpus = 2.0
+min_memory_gb = 4.0
+min_gpu_memory_gb = 1.0
+gpu_required = true
+gpu_class = "any"
+recommended_gpu = "any 8 GB+ NVIDIA GPU with Vulkan (RTX 4090 / L4)"
+verifier_min_cpus = 2.0
+verifier_min_memory_gb = 3.0
+verifier_min_gpu_memory_gb = 1.0
+measured = "50 oracle trials, 25 per mode (rcb_monitor samples every 5 s, eai-pandadyn:0.1.0, RTX 4090, 2026-10-13): privileged trial peak 0.79 GiB (anon + shmem), GPU 0.05 GiB; standard sim service peak 0.94 GiB, GPU 0.35 GiB, agent container 0.03 GiB; CPU p90 about 1 core per container; one filmed scene (render_video, S12 and D07): RSS 1.2-1.3 GiB, GPU 0.25 GiB; CPU: one core per scene (CPU PhysX, single-threaded) + one for the agent"
+
+# pandadyn, standard: 25 task(s)
+[[rule]]
+match = "^pandadyn-.*-standard$"
+min_cpus = 2.0
+min_memory_gb = 4.0
+min_gpu_memory_gb = 1.0
+gpu_required = true
+gpu_class = "any"
+recommended_gpu = "any 8 GB+ NVIDIA GPU with Vulkan (RTX 4090 / L4)"
+verifier_min_cpus = 2.0
+verifier_min_memory_gb = 3.0
+verifier_min_gpu_memory_gb = 1.0
+measured = "50 oracle trials, 25 per mode (rcb_monitor samples every 5 s, eai-pandadyn:0.1.0, RTX 4090, 2026-10-13): privileged trial peak 0.79 GiB (anon + shmem), GPU 0.05 GiB; standard sim service peak 0.94 GiB, GPU 0.35 GiB, agent container 0.03 GiB; CPU p90 about 1 core per container; one filmed scene (render_video, S12 and D07): RSS 1.2-1.3 GiB, GPU 0.25 GiB; the sim service is one scene, the collect hook replays in fresh processes one after the other; CPU: one core per scene + one for the agent"
+
[[limit]]
match = "^robofunolympics-.*-privileged$"
scope = "trial"
@@ -1752,6 +1780,36 @@ trial_max_gpu_memory_gb = 16.0
trial_max_memory_gb = 27.0
measured = "GPU: 10.4 GiB on a 48 GB card, 6.5 on 24 GB (batch_eval_live vram_profile (per-trial GPU peaks by card, 10-03..10)) -> max(1.5 x peak, peak + 4), at least 16 for Isaac Sim; memory: agent container docker peak 1.5 GiB over 1 samples (mem_guard (docker memory incl. page cache, top-4 containers every 10 min, ws-1, 10-02..10)) -> max(1.5 x peak, peak + 4, 8), at most the 24 GiB cap; whole trial: + the sim service's 19 GiB (docker peak 12.4 GiB, 1 samples); processes 512 and disk 20 GiB: design defaults, not yet measured"
+[[limit]]
+match = "^pandadyn-.*-privileged$"
+scope = "trial"
+max_gpu_memory_gb = 5.0
+max_memory_gb = 16.0
+max_processes = 512
+max_disk_gb = 10.0
+disk_paths = ["/app", "/tmp", "/root"]
+disk_exclude = []
+sustain_sec = 60.0
+window_sec = 120.0
+trial_max_gpu_memory_gb = 5.0
+trial_max_memory_gb = 16.0
+measured = "GPU: 0.25 GiB per scene -> max(1.5 x peak, peak + 4) with room for a few scenes at once; memory: 1.3 GiB per scene, 16 GiB lets an agent run about ten scenes in parallel (50 oracle trials, 25 per mode (rcb_monitor samples every 5 s, eai-pandadyn:0.1.0, RTX 4090, 2026-10-13): privileged trial peak 0.79 GiB (anon + shmem), GPU 0.05 GiB; standard sim service peak 0.94 GiB, GPU 0.35 GiB, agent container 0.03 GiB; CPU p90 about 1 core per container; one filmed scene (render_video, S12 and D07): RSS 1.2-1.3 GiB, GPU 0.25 GiB); processes 512 and disk 10 GiB: design defaults, not yet measured"
+
+[[limit]]
+match = "^pandadyn-.*-standard$"
+scope = "agent"
+max_gpu_memory_gb = 0.0
+max_memory_gb = 8.0
+max_processes = 512
+max_disk_gb = 10.0
+disk_paths = ["/app", "/tmp", "/root"]
+disk_exclude = []
+sustain_sec = 60.0
+window_sec = 120.0
+trial_max_gpu_memory_gb = 4.0
+trial_max_memory_gb = 14.0
+measured = "memory: the agent container holds only Python and the client (0.03 GiB in the oracle trials): max(1.5 x peak, peak + 4, 8) -> 8 GiB; whole trial: + the sim service's 6 GiB (one scene, 1.3 GiB, with its frames and the replays' scenes); GPU 0.25 GiB in the sim service (50 oracle trials, 25 per mode (rcb_monitor samples every 5 s, eai-pandadyn:0.1.0, RTX 4090, 2026-10-13): privileged trial peak 0.79 GiB (anon + shmem), GPU 0.05 GiB; standard sim service peak 0.94 GiB, GPU 0.35 GiB, agent container 0.03 GiB; CPU p90 about 1 core per container; one filmed scene (render_video, S12 and D07): RSS 1.2-1.3 GiB, GPU 0.25 GiB); processes 512 and disk 10 GiB: design defaults, not yet measured"
+
[[cap]]
match = "^autobio-.*-privileged$"
main = { cpus = 4.0, memory_gb = 12.0, pids = 8192 }
@@ -2027,3 +2085,12 @@ main = { cpus = 8.0, memory_gb = 16.0, pids = 8192 }
match = "^robofunolympics-.*-standard$"
main = { cpus = 4.0, memory_gb = 10.0, pids = 8192 }
sim = { cpus = 4.0, memory_gb = 8.0, pids = 8192 }
+
+[[cap]]
+match = "^pandadyn-.*-privileged$"
+main = { cpus = 8.0, memory_gb = 20.0, pids = 8192 }
+
+[[cap]]
+match = "^pandadyn-.*-standard$"
+main = { cpus = 4.0, memory_gb = 10.0, pids = 8192 }
+sim = { cpus = 4.0, memory_gb = 8.0, pids = 8192 }