Literature
Display
← All papers

DuoBench: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World

Tobias Jülg · Seongjin Bien · Simon Hilber · Yannik Blei · Pierre Krack · Maximilian Li · Sven Parusel · Rudolf Lioutikov · Florian Walter · Wolfram Burgard

Preprint posted · arXiv (preprint) ↗

Robotics10 min text · CondensedOriginal paper ↗
Contents

Overview

Project page. This edition follows the complete twenty-one-page, 2 November 2026 v2, including all eight figures, six numbered tables, four numbered equations, task recipes and 49 references.

Complete original PDF block S0.F1
Figure 1: Overview of DuoBench: four bimanual task categories with eleven tasks, four of which are replicated in the real world (left: real-world setup and simulated MuJoCo scene). Each task is decomposed into task stages to better understand policy failure modes. We provide a sim-to-real teleoperation pipeline to facilitate data collection and support extensibility.

At a glance

DuoBench evaluates dual-arm manipulation on eleven simulated tasks and four corresponding physical tasks. Its main contribution is diagnostic: monotonic task stages, conditional transition pass rates and confidence intervals expose where policies fail even when final success is rare. Fifty demonstrations per task/domain support comparisons of ACT, π₀.₅ and X-VLA. Early interaction, coordinated object motion and independent use of both arms remain difficult under this fixed setup and training protocol.

1 Introduction

Binary success obscures the difference between failing to grasp, transfer an object or finish a coordinated motion. DuoBench combines a functional taxonomy, reproducible simulation/physical task recipes, teleoperated datasets and stage annotations. Reproducibility means other labs can recreate tasks and collect comparable data; it does not claim a trained policy will transfer unchanged between laboratories.

2 Related Work

Manipulation benchmarks vary in simulator, task scope, sensing and evaluation. DuoBench builds on this infrastructure while emphasizing different coordination roles for two arms and repeatable task definitions across domains.

Single-Arm Manipulation in Simulation

RLBench, Meta-World, ManiSkill and LIBERO provide diverse manipulation environments. Household, long-horizon and multimodal benchmarks broaden the challenge; unified interfaces ease model comparisons. Their breadth motivates a complementary analysis of which bimanual interactions fail.

Dual-Arm Manipulation in Simulation

TWIN/RLBench2, RoboTwin, BiCoord and dexterous/humanoid benchmarks cover two-arm tasks with different emphases. RoboEval already uses stagewise progress plus coordination, efficiency and safety metrics. DuoBench’s contribution is the particular combination of coordination categories, stage-conditional diagnostics and reproducible physical counterparts, not the invention of all stage-based evaluation.

Real-World and Hybrid Benchmarks

FMB and FurnitureBench use reproducible objects, while RoboMind supplies multi-embodiment data and RoboArena distributes physical evaluation. DuoBench prioritizes dual-arm task recipes and a common collection/inference stack, spanning coordination patterns rather than only object substitutions.

3 Methodology

The Robot Control Stack represents environments as wrapped Markov decision processes. Wrappers modify observations/actions, record demonstrations and expose task progress. The formal dataset pairs actions with delayed camera/proprioceptive observations, including an instruction and head/left-wrist/right-wrist images. Joint-space actions have sixteen coordinates: seven joints and one gripper command per arm.

The learning objective is behavior cloning. Raw teleoperation supplies Cartesian commands, later converted to joint targets by observation-initialized inverse kinematics; the abstract sixteen-dimensional interface should be read together with this conversion pipeline.

Equations 1–2
𝒟={{(at,I,pt+1​Chead,t+1,Cright_wrist,t+1,Cleft_wrist,t+1)}t=1li}i=1N.\displaystyle\mathcal{D}=\{\{(a_{t},I,p_{t+1}C_{\text{head},t+1},C_{\text{right\_wrist},t+1},C_{\text{left\_wrist},t+1})\}_{t=1}^{l_{i}}\}_{i=1}^{N}. (1)
arg⁡maxθ​𝔼(a,s)∼𝒟​[log⁡πθ​(a|s)].\displaystyle\arg\max_{\theta}\mathbb{E}_{(a,s)\sim\mathcal{D}}[\log\pi_{\theta}(a|s)]. (2)

3.1 Task Stages

Task stages are milestones that can only increase. Conditions on the simulator state advance the recorded stage; independent-arm subtasks are merged with “at least one” and “all” conditions. Mean stage progress and the distribution of each episode’s highest stage remain informative when no complete task succeeds. Because progress is monotonic, losing an earlier physical configuration does not reduce its historical stage.

Conditional pass rate ρₖ asks how often an episode that reached stage k also reached k+1. It avoids hiding difficult late transitions behind early failures. Final success factors into these conditional probabilities without an independence assumption; the smallest defined pass rate names the bottleneck. If no episode reaches a stage, its later rate is undefined rather than evidence of zero competence.

Wilson 95% intervals quantify finite-rollout uncertainty. They do not capture variation across training seeds, task designs or laboratories. The stage count is a hand-designed progress measure, so equal normalized increments need not represent equal physical difficulty.

Equations 3–4
staget+1=max({staget}∪{h(∧ick,i(st),k):∀k∈K}).\displaystyle\text{stage}_{t+1}=\max(\{\text{stage}_{t}\}\cup\{h(\wedge_{i}c_{k,i}(s_{t}),k):\forall k\in K\}). (3)
ρk=P(κ≥k+1∣κ≥k),k=0,…,n−1,\displaystyle\rho_{k}=P\bigl(\kappa\geq k+1\mid\kappa\geq k\bigr),\qquad k=0,\ldots,n-1, (4)
pt=𝔼e​[staget(e)]max⁡Kp_{t}=\frac{\mathbb{E}_{e}[\text{stage}_{t}^{(e)}]}{\max K}
pfinal=𝔼e​[maxt∈ℰ⁡(e)⁡staget(e)]max⁡Kp_{\text{final}}=\frac{\mathbb{E}_{e}[\max_{t\in\mathcal{E}(e)}\text{stage}_{t}^{(e)}]}{\max K}
P⁡(maxt∈ℰ⁡(e)⁡staget(e)=k)P(\max_{t\in\mathcal{E}(e)}\text{stage}_{t}^{(e)}=k)
P⁡(maxt∈ℰ⁡(e)⁡staget(e)=max⁡K)P(\max_{t\in\mathcal{E}(e)}\text{stage}_{t}^{(e)}=\max K)
P⁡(κ=max⁡K)=∏kρkP(\kappa=\max K)=\prod_{k}\rho_{k}
arg⁡mink⁡ρk\arg\min_{k}\rho_{k}

3.2 Franka Duo Setup

Two Franka FR3 arms carry Robotiq 2F-85 grippers. A Zed Mini main camera and two RealSense D405 wrist cameras provide three views. Mounts follow the manufacturer’s arrangement and specifications; the matching MuJoCo scene uses Menagerie and manufacturer assets. This is one fixed embodiment/sensing configuration.

Complete original figure S3.F2
Figure 2: FR3 Duo setup. Left shows the real-world setup with printed assets. Right shows the simulated MuJoCo scene.
Complete original PDF block S3.T1
Table 1: Overview of all eleven tasks in our four bimanual task categories. (nn) indicates the number of stages for each task. Tasks above the gray line are implemented in simulation and real-world settings. Tasks below are only implemented in simulation.
Original selectable table
Asymmetric Support Bimanual Manipulation Sequential Handoff Parallel Execution
Hinge-Chest (3) Ball-Maze (4) Transfer-Cube (4) Bin-Sort (5)
Spring-Door (3) Carry-Pot (3) Transfer-Reorient (4)
Pour-Marbles (6) Block-Balance (5) Transfer-Gate (4)
Join-Blocks (3)

3.3 Bimanual Task Taxonomy

The taxonomy adapts Krebs and Asfour to four functional categories: asymmetric support, where one arm creates conditions for the other; joint manipulation of the same object through a closed kinematic chain; sequential handoff; and independent parallel execution. The last category permits simultaneous or sequential solutions and does not require direct arm interdependence.

Eleven tasks instantiate these roles. Hinge-Chest, Ball-Maze, Transfer-Cube and Bin-Sort are also reproduced physically, one per category. Each task randomizes at least object placement; the appendix retains exact stages, horizon statistics, printability and randomization. The taxonomy is a useful selected test set, not exhaustive coverage of bimanual behavior.

3.4 Task Composition, Data Collection and Evaluation

A MuJoCo MJSpec composer creates scenes; reset and stage wrappers access simulator state to randomize tasks and score milestones. Environments register with Gymnasium. Physical assets follow the same recipes, but a human operator judges stages on hardware.

The IRIS VR interface supports simulated and physical teleoperation. Recording simulator states enables replay under new visuals without new demonstrations. VLAgents runs policies and records trials with seeded initialization. Such replay is an available tool, not an augmentation used in the reported experiments.

4 Results

The comparison uses ACT, π₀.₅ and X-VLA. All execute thirty absolute joint-target actions before the next observation/prompt: one second open loop at 30 Hz. Trials start from a common home pose, with object positions drawn from the demonstration distribution. Simulation resets share a seed; physical placements are matched manually. Conclusions therefore depend on this observation cadence and in-distribution setup.

Complete original PDF block S4.T2
Table 2: DuoBench simulation evaluation, 300 rollouts per cell. Each cell reports the success rate (SR, %) with its Wilson 95% interval, the mean normalized task progress pfinalp_{\text{final}}, and the bottleneck stage with its pass rate ρk\rho_{k}.
Original selectable table
Task ACT π0.5\pi_{0.5} X-VLA
SR [CI] pfinalp_{\text{final}} bottleneck stage (ρk\rho_{k}) SR [CI] pfinalp_{\text{final}} bottleneck stage (ρk\rho_{k}) SR [CI] pfinalp_{\text{final}} bottleneck stage (ρk\rho_{k})
Asymmetric Support
Hinge-Chest 26.7 [22,32] 0.55 cube + lid (0.39) 26.7 [22,32] 0.39 lid or cube (0.56) 12.0 [9,16] 0.30 cube + lid (0.37)
Spring-Door 22.3 [18,27] 0.50 box + door (0.33) 19.7 [16,25] 0.44 box + door (0.39) 13.7 [10,18] 0.38 box + door (0.37)
Pour-Marbles 0.3 [0,2] 0.12 pour all (0.23) 0.0 [0,1] 0.08 pour all (0.00) 0.0 [0,1] 0.07 place cups (0.00)
Bimanual Manipulation
Carry-Pot 54.0 [48,60] 0.70 grasp + lift (0.55) 26.0 [21,31] 0.51 grasp + lift (0.30) 12.3 [9,17] 0.40 grasp + lift (0.14)
Ball-Maze 8.0 [5,12] 0.71 ball to goal (0.09) 5.0 [3,8] 0.67 ball to goal (0.06) 2.0 [1,4] 0.71 ball to goal (0.02)
Block-Balance 0.0 [0,1] 0.12 grasp rects (0.00) 0.0 [0,1] 0.06 grasp rects (0.00) 0.0 [0,1] 0.15 release (0.00)
Join-Blocks 2.3 [1,5] 0.17 connect (0.06) 0.0 [0,1] 0.16 to wall (0.00) 0.0 [0,1] 0.20 to wall (0.00)
Sequential Handoff
Transfer-Cube 10.0 [7,14] 0.14 pick cube (0.20) 13.0 [10,17] 0.28 pick cube (0.47) 9.3 [7,13] 0.17 pick cube (0.33)
Transfer-Gate 42.7 [37,48] 0.49 pick box (0.59) 23.3 [19,28] 0.35 pick box (0.49) 26.3 [22,32] 0.36 pick box (0.51)
Transfer-Reorient 0.3 [0,2] 0.35 insert (0.01) 1.3 [1,3] 0.36 insert (0.04) 0.0 [0,1] 0.30 insert (0.00)
Parallel Execution
Bin-Sort 0.7 [0,2] 0.11 pick 2nd (0.09) 1.7 [1,4] 0.14 pick 2nd (0.16) 1.0 [0,3] 0.13 pick 2nd (0.21)

4.1 Datasets

Fifty demonstrations for each of eleven simulated and four physical tasks yield 750 episodes. Raw recordings contain three 1,280×720 camera streams, joint/Cartesian states, instructions and domain-specific robot/simulator state. Conversion to LeRobot resizes images to 224×224, flattens state/action vectors and resolves Cartesian commands into joint targets.

The source reports 442,907 frames and an average of about 350 frames/12 seconds per episode. Dividing the stated total by 750 instead gives about 591 frames/19.7 seconds at 30 Hz; no distinction between these denominators is explained. The original figures are preserved rather than silently reconciled.

4.2 Evaluation in Simulation

ACT trains a separate policy on fifty demonstrations per task; the VLAs share one instruction-conditioned policy trained on all 550 simulated episodes. Each task/policy receives 300 rollouts, with per-task cutoffs calibrated from demonstrations and no image/replay augmentation. This compares the stated specialist and multitask training regimes, not equal total data or training compute.

ACT reaches 54.0% on Carry-Pot and 42.7% on Transfer-Gate, versus π₀.₅’s 26.0%/23.3% and X-VLA’s 12.3%/26.3%. Transfer-Cube is much lower at 10.0%/13.0%/9.3%; the authors suggest the gate constrains the handover configuration, but the cross-task contrast alone does not isolate that cause.

All three score zero on Block-Balance; Bin-Sort success is 0.7%/1.7%/1.0%. Seventeen of 33 task-policy combinations have a grasp-establishment bottleneck. Carry-Pot and Ball-Maze instead bottleneck after establishing contact, when both arms must move the shared object. Ball-Maze shows why progress matters: normalized progress 0.67–0.71 coexists with only 2–8% final success.

Unconditional failure-stage bars and conditional pass-rate profiles answer different questions. X-VLA can acquire the Transfer-Reorient peg more often yet struggle with handoff; Table 2 identifies final insertion as the minimum conditional pass rate. Every profile is retained in Appendix A.

Complete original PDF block S4.F3
Figure 3: Fraction of rollouts in simulation that failed in a given stage. Success rates are annotated above. We abbreviate policy names as follows: ACT as “A”, π0.5\pi_{0.5} as “π\pi” and X-VLA as “X”.

4.3 Evaluation in Real-World

Physical evaluation retrains ACT per task and the VLAs jointly on the 200 physical demonstrations, then tests fifteen rollouts per task/policy. All three fail Hinge-Chest. π₀.₅ completes Transfer-Cube in 6/15 trials (40%) and Bin-Sort in 3/15 (20%); X-VLA completes Ball-Maze in 3/15, while ACT completes Ball-Maze and Bin-Sort once each.

Grasping and shared-object motion recur as difficulties across domains, but exact rankings and rates differ. Ball-Maze policies often establish the two-arm grasp without guiding the ball successfully. Transfer-Cube acquisition is poor for ACT/X-VLA on hardware, whereas π₀.₅ picks reliably enough that handover becomes its bottleneck. Four tasks with fifteen trials each support qualitative diagnostic comparisons, not a general simulation-to-reality calibration guarantee.

4.4 Mixed Training and Sim-to-Real Gap

Mixed ACT training uses one hundred demonstrations per replicated task; mixed VLA training uses all 750 episodes. Table 3 compares domain-specific policies on their own domain with mixed policies on both domains. It does not report a simulation-only policy evaluated zero-shot on hardware.

Real-world average success changes from about 3% to 7% for ACT, while π₀.₅ remains 15% and X-VLA remains 5%, with task-level trade-offs. π₀.₅ improves Transfer-Cube from 40% to 53% but falls on Bin-Sort from 20% to 7%; X-VLA falls on Ball-Maze from 20% to 7%. These changes involve only a few successes in fifteen trials.

Across the four replicated simulation tasks, mixed training reduces mean success from 11.3% to 9.8% for ACT, 11.6% to 4.8% for π₀.₅ and 6.1% to 3.6% for X-VLA. X-VLA shares one domain embedding across simulation and reality, a proposed explanation rather than an ablated cause. Naive mixing does not consistently improve both domains, and physical mean progress can decline even when average success is unchanged.

Complete original PDF block S4.T3
Table 3: Evaluation of domain independent training vs mixed-data training. Each cell reports success rate with mean normalized task progress in parentheses for real and simulation evaluation.
Original selectable table
Model ACT π0.5\pi_{0.5} X-VLA
Train dataset Sim Real Mixed Sim Real Mixed Sim Real Mixed
Eval domain Sim Real Sim Real Sim Real Sim Real Sim Real Sim Real
Hinge-Chest 26.7 (0.55) 0 (0.04) 26.7 (0.56) 0 (0.18) 26.7 (0.39) 0 (0.00) 4.7 (0.08) 0 (0.07) 12.0 (0.30) 0 (0.13) 8.7 (0.19) 0 (0.04)
Ball-Maze 8.0 (0.71) 7 (0.52) 3.7 (0.69) 13 (0.43) 5.0 (0.67) 0 (0.50) 6.0 (0.69) 0 (0.30) 2.0 (0.71) 20 (0.60) 1.3 (0.66) 7 (0.12)
Transfer-Cube 10.0 (0.14) 0 (0.00) 8.3 (0.13) 0 (0.00) 13.0 (0.28) 40 (0.53) 8.0 (0.23) 53 (0.60) 9.3 (0.17) 0 (0.02) 4.0 (0.15) 7 (0.13)
Bin-Sort 0.7 (0.11) 7 (0.28) 0.7 (0.15) 13 (0.37) 1.7 (0.14) 20 (0.47) 0.7 (0.14) 7 (0.29) 1.0 (0.13) 0 (0.21) 0.3 (0.10) 7 (0.20)
Overall 11.3 (0.38) 3 (0.21) 9.8 (0.38) 7 (0.25) 11.6 (0.37) 15 (0.37) 4.8 (0.28) 15 (0.31) 6.1 (0.33) 5 (0.24) 3.6 (0.28) 5 (0.12)
Complete original PDF block S4.F4
Figure 4: Stage-conditional pass rates ρk\rho_{k} with Wilson 95% intervals for the four tasks that exist in simulation (top, 300 rollouts) and the real world (bottom, 15 rollouts), each policy trained on the data of the evaluated domain. The xx-axis names the sub-instruction to leave stage kk. A series ends where no episode reached the next stage.

5 Limitations

Physical stage labels require human judgment; taxonomy and milestone definitions reflect designer choices. Evaluation uses one hardware/camera setup, a small demonstration regime, one default training seed and only fifteen physical trials per cell. Neither inter-lab transfer, visual robustness nor broader embodiment coverage is established. The common one-second open-loop action interval also limits how directly these results generalize to more reactive controllers.

6 Conclusion

DuoBench supplies reproducible dual-arm tasks and diagnostic measurements that distinguish grasp acquisition, coordinated object motion, transfer and parallel action failures. Its data show substantial headroom under the tested training and control protocol. More task varieties, tactile-relevant challenges, richer sensing and improved cross-domain training remain future directions.

Acknowledgments

The authors acknowledge assistance with physical evaluation and support from GeniusRobot, the German research ministries, Robotics Institute Germany and NHR@FAU/BayernKI computing resources.

Appendix A Extended Results

A.1 Task Progress over Time

Figure 5 plots average reached stage over normalized rollout time for all eleven simulated tasks. Early rises indicate earlier milestone acquisition and plateaus indicate stalled progress. The recorded stage never decreases, so these curves cannot directly measure the loss of an already achieved physical configuration. Time normalization also prevents reading the horizontal axis as equal wall-clock duration across tasks.

Complete original PDF block A1.F5
Figure 5: Average task progress over normalized time across all rollouts in simulation.

A.2 Failure-Stage Distributions

The physical failure-stage bars retain the distribution of highest reached stage over fifteen trials, with green indicating complete success. They expose how early attrition differs across policies, while remaining distinct from conditional pass probabilities among episodes that reached a particular milestone.

Complete original PDF block A1.F6
Figure 6: Fraction of real-world rollouts that ended in a given stage. Green means fraction of successful episodes annotated with the success rate. Policy names are abbreviated as follows: ACT as “A”, π0.5\pi_{0.5} as “π\pi” and X-VLA as “X”.

A.3 Pass-Rate Profiles

All eleven simulated pass-rate profiles include Wilson intervals and named outgoing subinstructions. Curves stop when no rollout reaches the next stage. Late-stage estimates may be based on far fewer than 300 examples and can have wide intervals even though the initial evaluation budget is the same.

Complete original PDF block A1.F7
Figure 7: Stage-conditional pass rates ρk\rho_{k} with Wilson 95% intervals for all eleven tasks in simulation (300 rollouts per policy and task). The xx-axis names the sub-instruction to leave stage kk; a series ends where no episode reached the next stage.

A.4 Real-World Results

Table 4 adds confidence intervals, progress and bottlenecks for real-only training. Zero successes out of fifteen has a Wilson interval rounded to 0–20%; six successes gives 40% with interval 20–64%. These broad intervals matter when interpreting apparently large differences between physical policies.

Complete original PDF block A1.T4
Table 4: Real-world results, 15 rollouts per cell, policies trained on real data only: success rate (SR, %) with Wilson 95% interval, pfinalp_{\text{final}}, and the bottleneck stage with its pass rate ρk\rho_{k}.
Original selectable table
Task ACT π0.5\pi_{0.5} X-VLA
SR [CI] pfinalp_{\text{final}} bottleneck stage (ρk\rho_{k}) SR [CI] pfinalp_{\text{final}} bottleneck stage (ρk\rho_{k}) SR [CI] pfinalp_{\text{final}} bottleneck stage (ρk\rho_{k})
Hinge-Chest 0 [0,20] 0.04 cube + lid (0.00) 0 [0,20] 0.00 lid or cube (0.00) 0 [0,20] 0.13 cube + lid (0.00)
Ball-Maze 7 [1,30] 0.52 ball to goal (0.10) 0 [0,20] 0.50 ball to goal (0.00) 20 [7,45] 0.60 ball to goal (0.27)
Transfer-Cube 0 [0,20] 0.00 pick cube (0.00) 40 [20,64] 0.53 hand over (0.67) 0 [0,20] 0.02 both grippers (0.00)
Bin-Sort 7 [1,30] 0.28 pick 2nd (0.17) 20 [7,45] 0.47 pick 2nd (0.44) 0 [0,20] 0.21 place 2nd (0.00)

Appendix B Training Details

LeRobot v0.5.1 and Accelerate train with bfloat16, one observation step, a default seed and no image augmentation. ACT uses an ImageNet-pretrained ResNet-18, 150,000 steps, effective batch 64 and one H100, without temporal ensembling. π₀.₅ and X-VLA fully fine-tune their public checkpoints for 40,000 steps: effective batches 64/192 on four H200/four H100 GPUs. Vision/language encoders remain trainable.

Table 5 preserves optimizer, schedule, clipping and normalization details. Chunk lengths are 100/50/30, but all execute only thirty actions before replanning. Joint targets are clipped to arm limits and gripper commands binarized. X-VLA zero-pads the sixteen-dimensional action to twenty dimensions, uses squared-error loss on actual action dimensions including grippers, and shares its domain embedding across datasets. Different pretraining, update counts and compute complicate model-only causal comparisons.

Complete original PDF block A2.T5
Table 5: Training hyperparameters. Batch size is per GPU; the effective batch size is given in parentheses.
Original selectable table
ACT π0.5\pi_{0.5} X-VLA
Initialization from scratch (ImageNet ResNet-18) lerobot/pi05_base lerobot/xvla-base
Trainable parameters all all (VLM incl. vision encoder, action expert) all (encoders, transformer, soft prompts)
Training regime one policy per task one policy for all tasks one policy for all tasks
Training steps 150 000 40 000 40 000
Batch size 64 (64) 16 (64) 48 (192)
Optimizer AdamW, β=(0.9,0.999)\beta=(0.9,0.999) AdamW, β=(0.9,0.95)\beta=(0.9,0.95) AdamW, β=(0.9,0.99)\beta=(0.9,0.99)
Learning rate 1×10−51\times 10^{-5}, constant 2.5×10−52.5\times 10^{-5}, cosine decay 1×10−41\times 10^{-4}, cosine decay
Warmup / decay steps – 1 000 / 30 000 (to 2.5×10−62.5\times 10^{-6}) 1 000 / 30 000 (to 2.5×10−62.5\times 10^{-6})
Weight decay 10−410^{-4} 0.01 0
Gradient clipping 10.0 1.0 10.0
Action chunk size 100 50 30
Normalization (state / action) mean-std / mean-std quantiles / quantiles none / mean-std
Inference flow / denoising steps – 10 10
Precision bf16 bf16 bf16
GPUs 1×\times H100 4×\times H200 4×\times H100

Appendix C Visual Randomization

The replayer restores recorded simulator states and renders new backgrounds, textures and lighting. Figure 8 retains twenty example views, including MolmoSpaces scenes and combined changes. Instruction-referenced objects retain their appearance. These are capability demonstrations; no reported model is trained or evaluated under these visual shifts.

Complete original figure A3.F8
Figure 8: Visual randomization of one Hinge-Chest frame from the head camera. Row 1: default scene (left) and MolmoSpaces [19] backgrounds. Row 2: table and chest textures. Row 3: lighting. Row 4: all combined.

Appendix D 3D Printing and Assembly

Physical task assets are available for 3D printing. The reported setup uses a Bambulab X1C; Hinge-Chest attaches its lid with two M3×2 cm screws and nuts. The complete recipes support task recreation, while other setups still need their own calibration and comparable demonstrations.

Appendix E Tasks

The full three-page task catalog below preserves every scene image, description, stage sequence and randomization rule. “Steps” is the 99th-percentile demonstration length, not a success rate or universal execution budget.

Asymmetric support covers Hinge-Chest (hold a lid while inserting a cube; three stages), Spring-Door (hold a sprung microwave door; three) and Pour-Marbles (grasp/lift two cups, pour one/all marbles, replace cups; six). Randomization changes object positions and, for pouring, which cup contains marbles.

Shared-object tasks are Ball-Maze (contact, grasp, lift, guide the ball; four stages, ten board designs), Carry-Pot (contact handles, lift jointly, place on stove; three), Block-Balance (place a beam, load both ends and release without tipping; five) and Join-Blocks (grasp, connect, attach to a wall peg; three). Object positions vary; manipulation and contact constraints differ substantially across them.

Sequential tasks transfer a cube to a bowl, pass a box through a gate onto a mat, or hand over/reorient a peg for insertion; each has four stages with placements constrained by the two arm workspaces. Bin-Sort uses five milestones for acquiring and placing two cubes, allowing alternative execution orders. Only Hinge-Chest, Ball-Maze, Transfer-Cube and Bin-Sort have physical counterparts in this evaluation.

Complete original PDF block A5.T6.part1Complete original PDF block A5.T6.part2Complete original PDF block A5.T6.part3
Table 6: Overview of all benchmark tasks grouped into the categories of sequential handover, parallel execution, asymmetric support, and bimanual manipulation. The table summarizes whether tasks were replicated in a real-world setup, whether they are suitable for 3D printing, the 99th percentile episode length in steps as an indicator of task horizon, and the principal subtasks involved. Additionally, the employed randomization strategies are reported.
Original selectable table
Scene Task Description Category 3D Printable Real-World Steps Key Subtasks Randomization
[Uncaptioned image] Hinge-Chest Holding the lid of a small chest open while inserting a cube. One arm must hold the lid while the other inserts the cube. Asymmetric Support ✓ ✓ 441441 open the chest OR pick up the cube, pick up the cube AND open the chest, place the cube inside the chest cube position, chest position
[Uncaptioned image] Spring-Door A spring-loaded microwave door requires one arm to hold it open while the other inserts a box. Asymmetric Support ×\times ×\times 807807 open the microwave OR pick up the box, pick up the box AND open the microwave, place the box inside the microwave box position, microwave position
[Uncaptioned image] Pour-Marbles Two cups, one containing marbles. Both cups must be picked up, and the marbles must be poured into the other cup before both cups are placed back. Asymmetric Support ×\times ×\times 442442 grasp at least one cup, grasp both cups, lift both cups, pour at least one marble into the target cup, pour all marbles into the target cup, place both cups cup position, which cup contains marbles
[Uncaptioned image] Ball-Maze Pick up a maze board with both arms and tilt it so a ball rolls into a target region. Bimanual Manipulation ✓ ✓ 350350 make contact with the maze, grasp the maze with both arms, lift the maze and move the ball out of the start area, guide the ball from the start area toward the goal board position, one out of 10 different boards selected
[Uncaptioned image] Carry-Pot Carry a pot using both side handles and place it on a stove. Both arms are needed to lift the pot. Bimanual Manipulation ×\times ×\times 450450 make contact with a pot handle, grasp the pot by both handles and lift it, place the pot onto the stove stove and pot positions within one half of the table each
[Uncaptioned image] Block-Balance Consists of a red support cube, a beam, and two additional rectangular blocks. The beam needs to be placed on the cube, and then both blocks need to be placed on the beam simultaneously. Bimanual Manipulation ✓ ×\times 766766 grasp the beam with one arm, place the beam on the small cube, grasp both rectangles, place both rectangles onto the beam, release and retract while keeping the beam balanced position of all blocks
[Uncaptioned image] Join-Blocks Connect two movable blocks together and then attach them to a peg on a third stationary block. Bimanual Manipulation ✓ ×\times 829829 approach and grasp both blocks, connect the two blocks, connect the assembled blocks to the wall position of both movable blocks, each on one half of the table
[Uncaptioned image] Transfer-Cube Hand over a cube between arms before placing it into a bowl. Sequential Handoff ✓ ✓ 441441 pick up the cube, bring both grippers into contact with the cube, transfer the cube to the other gripper, place the cube correctly and release it cube and bowl positions, each on one half of the table
[Uncaptioned image] Transfer-Gate Hand over a box between arms before placing it onto a mat. The box has to be passed through a gate. Sequential Handoff ✓ ×\times 523523 pick up the box, pass the box through the ring, grab the box with the other hand, place the box on the mat box, mat and goal positions, box and mat on each half of the table while the gate is in between
[Uncaptioned image] Transfer-Reorient The right arm picks up a peg and hands it over to the left arm in such a way that the left arm is then able to insert it into a socket. Sequential Handoff ✓ ×\times 505505 pick up the peg, bring both grippers into contact with the peg, transfer the peg to the other gripper, insert the peg into the matching socket peg and socket positions, each on one half of the table
[Uncaptioned image] Bin-Sort Sort two cubes into matching bowls, testing simultaneous execution rather than direct cooperation. Parallel Execution ✓ ✓ 216216 pick up at least one cube, pick up the other cube or place the picked cube in the correct bowl, place at least one cube in the correct bowl, pick up the second cube, place the second cube in the correct bowl cube and bowl positions
Concept demonstration

Open image ↗