Literature
Display
← All papers

WireCraft: A Simulation Benchmark for Industrial DLO Manipulation

Chongyu Zhu · Ramy ElMallah · Hyegang Kim · Zachary Tang · Jiachen Rao · Artem Arutyunov · Seungyeon Ha · Chi-Guhn Lee

Preprint posted · CoRL 2026 (accepted) ↗

Robotics15 min text · CondensedOriginal paper ↗
Contents

Overview

Project repository. This edition follows the complete twenty-nine-page, 16 October 2026 v3, including all twelve figures, thirteen PDF tables and the teleoperation action display. The benchmark-comparison table missing from the arXiv HTML is restored from PDF page 16; displayed table numbering follows the PDF.

At a glance

WireCraft benchmarks fixture-constrained wire manipulation: inserting a connector, routing a cable through clips and seating its body in a channel. It combines configurable assets, articulated and FEM wire models, common data interfaces and physical UR5 validation. Privileged state policies solve selected simulated settings above 82%; image-based baselines are substantially weaker. Adding simulation data to real demonstrations improves the best observed hardware result by 15–65 percentage points across tasks, but this is data-assisted adaptation rather than demonstrated zero-shot transfer.

1 Introduction

Industrial wire handling requires both precise connector poses and control of a deforming body that interacts with fixtures, the table and itself. Existing general manipulation suites, deformable-object simulators and physical task boards cover different parts of this problem. WireCraft joins three industrial task families, printable fixtures, compatible demonstrations and shared evaluation criteria.

The contribution is benchmark infrastructure and a reference study, not a new universally reliable manipulation policy. Configurable difficulty and three supported simulated arms extend the available task space beyond the narrower settings actually evaluated.

Complete original PDF block S1.F1
Figure 1: WireCraft simulates industrial DLO manipulation across three task families: connector insertion, clip routing, and channel seating with trajectories from multiple sources. Tasks run on the UR5, Franka, and Trossen Stationary AI in simulation, with real-world validation on a UR5.

2 Related Work

General suites such as RLBench and ManiSkill3 contrast with deformable-object environments including SoftGym and DaXBench. ReForm and DLO-Lab focus more specifically on flexible linear objects; physical boards and applied assembly studies provide complementary hardware protocols.

WireCraft emphasizes their combination: industrial insertion/routing/seating, two wire representations, multiple learning-policy families and simulation plus real demonstrations. The comparison table uses the authors’ feature definitions; its coverage markings are not an independent audit of every prior system. The evaluated methods are representative baselines rather than exhaustively optimized solvers.

3 WireCraft Infrastructure

Complete original PDF block S3.F2
Figure 2: WireCraft Architecture, including the WireCraft simulation engine, asset library, task suite, demonstration sources, policy baselines, and real-world validation setup. WireCraft supports articulated and FEM-based deformable wire models, provides configurable industrial DLO manipulation tasks across connector insertion, clip routing, and channel seating, and pairs simulation data with real-world teleoperation trajectories under a shared dataset schema.

3.1 WireCraft Engine

Isaac Lab 2.2.1 on Isaac Sim 4.5 supplies rendering, parallel environments and physics. Articulated wires connect rigid segments with passive six-degree-of-freedom joints; the FEM alternative models local bending, compression and contact deformation. The former supports high aggregate throughput but can suffer unstable self-contact.

RGB, optional depth, contact/force interfaces and proprioception are available, but available sensors are not necessarily policy inputs. The two wire models share task interfaces rather than calibrated dynamics; the paper does not quantitatively establish their relative real-cable accuracy.

3.2 WireCraft Assets

Wire length, radius and material or joint parameters are configurable. Printable cylindrical, cuboid and reference-style connectors vary rotational symmetry; sockets, clips and channels define fixture constraints. UR5, Franka and Trossen Stationary AI are supported in simulation, while reported hardware validation uses only UR5.

Complete original PDF block S3.F3
Figure 3: WireCraft Assets. Top: the two configurable wire representations, articulated and FEM-based deformable, shown as rendered DLOs (left) and their construction-level chain/mesh structure (right). Bottom: a subset of representative 3D-printable connectors and task fixtures.

4 WireCraft Benchmark

4.1 WireCraft Tasks

Insertion combines connector alignment with trailing-wire disturbance. Routing requires seating the wire through clip openings, and channel seating constrains its extended shape. Difficulty varies with connector geometry, clip count/placement and channel curvature, while per-episode initialization changes poses.

The experiment set covers four insertion geometries, one-clip routing and straight seating. USB-A, multi-clip and curved-channel configurations are available assets or curricula, not established policy-performance results here.

4.2 WireCraft Demonstrations

Scripted state-based controllers, RL rollouts, simulation teleoperation and physical demonstrations share a LeRobot 3.0 schema. For UR5, the principal state/action vectors each contain six arm values plus the gripper, alongside wrist/side RGB and a task instruction.

The teleoperation interface is released and illustrated, but simulated teleoperation trajectories are not included in the reported policy-training datasets. Its potential for correction-rich demonstrations is therefore a capability, not an evaluated data-source benefit.

5 Experiments

Insertion reports both reaching within 2 cm and actual tip seating; reaching is not successful assembly. Routing requires clip seating, and channel success requires more than 80% of the wire inside the channel. Reported standard deviations are across evaluation or RL seeds under the specified protocols, not confidence intervals for all possible hardware conditions.

5.1 Baselines on Connector Insertion

On Ethernet-style insertion with 3 mm clearance, privileged State PPO and SACfD insert at about 95.9% and 92.4%. Image-based ACT reaches/inserts at 60.9/29.7%, Diffusion Policy at 37.5/20.3%, Vision PPO at 51.6/17.7%, DiT VLA at 10.9/3.1% and π₀.₅ at 6.2/0%. Vision-based reaching itself can fail, and successful transport does not guarantee a suitable grasp or final alignment.

The methods differ in information, architectures, demonstration use and optimization budgets. Low VLA results with frozen vision components and limited fine-tuning are not an architecture-level impossibility result. The suggested pretraining-distribution explanation is not separately isolated; insertion IL/VLA checkpoint selection also uses evaluation reach scores.

Complete original PDF block S5.F4
Figure 4: Ethernet-style Connector Insertion performance. SRreach\mathrm{SR}_{\mathrm{reach}} and SRinsert\mathrm{SR}_{\mathrm{insert}} on the shared simulated insertion task show privileged RL solves the task reliably, while vision-based baselines exhibit a substantial gap between reaching the socket neighborhood and completing insertion.

5.2 RL Baselines on Diverse Connector Insertion

Across cylinder, cuboid, Ethernet-style and DisplayPort fixtures, State PPO insertion stays around 93–96%, while SACfD ranges from 77.43% to 93.06%. Vision PPO ranges from 3.32% on DisplayPort to 41.49% on the cylinder. The detailed table preserves both reach and insertion results.

Geometry is not a monotonic difficulty scale for every policy: Vision PPO handles Ethernet-style better than cuboid, and privileged methods do not always favor the cylinder. Fixed-orientation reward bias and shape-specific visual cues are plausible explanations, not controlled causal ablations. High state-based success demonstrates feasibility for these settings, not that control difficulty disappears generally.

Complete original PDF block S5.T1
Table 1: RL baseline success rates (%) on the diverse connector insertion tasks.
Original selectable table
Connector Metric State PPO SACfD Vision PPO
Cylinder SRreach\mathrm{SR}_{\mathrm{reach}} 95.00±1.2595.00\pm 1.25 97.27±1.0697.27\pm 1.06 68.07±6.0268.07\pm 6.02
SRinsert\mathrm{SR}_{\mathrm{insert}} 93.75±1.2593.75\pm 1.25 85.55±8.6885.55\pm 8.68 41.49±1.3241.49\pm 1.32
Cuboid SRreach\mathrm{SR}_{\mathrm{reach}} 97.08±1.4497.08\pm 1.44 96.70±1.0696.70\pm 1.06 41.67±4.7341.67\pm 4.73
SRinsert\mathrm{SR}_{\mathrm{insert}} 95.00±3.7595.00\pm 3.75 93.06±6.4693.06\pm 6.46 6.25±3.316.25\pm 3.31
Ethernet-style SRreach\mathrm{SR}_{\mathrm{reach}} 95.86±1.9395.86\pm 1.93 98.05±1.0598.05\pm 1.05 51.63±4.8251.63\pm 4.82
SRinsert\mathrm{SR}_{\mathrm{insert}} 95.86±1.9395.86\pm 1.93 92.40±5.9192.40\pm 5.91 17.74±1.2117.74\pm 1.21
DisplayPort SRreach\mathrm{SR}_{\mathrm{reach}} 95.00±1.2595.00\pm 1.25 95.62±10.5895.62\pm 10.58 34.01±5.7334.01\pm 5.73
SRinsert\mathrm{SR}_{\mathrm{insert}} 92.92±3.1592.92\pm 3.15 77.43±6.1777.43\pm 6.17 3.32±1.453.32\pm 1.45

5.3 Clip Routing and Channel Seating

State PPO succeeds at 95.32% on one-clip routing and 82.33% on straight seating. ACT reaches 64.06%/18.80%, and DiT VLA 39.06%/15.62%. These results extend the privileged-state versus visual-policy gap beyond insertion, but do not evaluate the harder multi-clip or curved-channel curricula.

Complete original PDF block S5.T2
Table 2: Evaluation results on 1-clip routing and straight channel seating tasks in simulation.
Original selectable table
Task Setting State PPO ACT DiT VLA
Clip routing 1-clip 95.32±0.93\bm{95.32\pm 0.93} 64.06±11.1664.06\pm 11.16 39.06±16.1639.06\pm 16.16
Channel seating Straight channel 82.33±0.31\bm{82.33\pm 0.31} 18.80±7.6518.80\pm 7.65 15.62±12.8815.62\pm 12.88

6 Real-World Validation

Hardware compares ACT trained on real data R, real plus scripted simulation R+S, real plus simulation RL R+P and all three R+S+P, with twenty trials per task/mixture. Insertion success is 15/45/65/80%; routing 45/50/65/60%; seating 75/85/90/90%. The strongest mixture is task-dependent; adding every source does not monotonically improve routing.

Insertion reach stays 80–95%, yet R+P reaches more often than R+S+P while seating less often. The observed grasp-pose differences suggest that a transportable grasp can be unsuitable for mating. Routing near-misses remain failures even when the wire lands closer to the clip; seating failures often originate at grasp acquisition.

All tested mixtures contain real demonstrations, after poor preliminary simulation-only transfer. The physical initialization is narrower than the simulation benchmark. Source composition and total data quantity change together, so the results demonstrate supplementation under this setup rather than isolating diversity or proving zero-shot transfer.

Complete original PDF block S6.F5
Figure 5: ACT success rates on the physical UR5 for (a) Ethernet-style connector insertion, (b) 1-clip routing, and (c) straight-channel seating. Insertion reports both SRreach\mathrm{SR}_{\mathrm{reach}} and SRinsert\mathrm{SR}_{\mathrm{insert}}; routing and seating each report task success.

7 Limitations

Neither wire model is calibrated to the physical cable, their dynamic/contact behavior is quantitatively unmatched, and cross-model policy transfer is untested. Optional depth and full force/torque feedback are not evaluated; privileged insertion uses saturated fingertip contacts, and the physical force/torque sensor is unread.

Only selected tasks on one UR5 and limited wire/fixture combinations receive hardware testing. Binary success omits contact loads and wire damage. VLA outcomes depend on constrained training, and small physical trial counts cannot establish a precise universal ordering of data mixtures.

8 Conclusion

The benchmark separates task feasibility with privileged state from the difficulty of visual localization, grasp selection, deformation and fixture engagement. Its common assets and data interface enable more comparable future studies, while the current hardware results support task portability with real-data adaptation rather than physically validated wire simulation.

Acknowledgments

The work acknowledges partial support from the Republic of Korea’s industrial technology innovation program for autonomous assembly of flexible and complex components.

Appendix A Related Work

The restored thirty-two-row comparison lists twelve feature dimensions for prior environments, applied systems and WireCraft. Its definitions distinguish insertion, fixture routing and extended channel placement, as well as learning baselines, robot evaluation and multiple demonstration sources. This is a categorical author comparison, not a measured performance ranking.

Complete original PDF block A1.T4-restored
Table 4: Comparison with related DLO and deformable-object benchmarks. WireCraft combines industrial DLO task families, articulated and FEM-based deformable wire simulation, RL/IL/VLA baselines, simulated-robot evaluation, and physical UR5 validation within a unified benchmark framework. Insertion, Routing, and Seating refer to end/part insertion, fixture-guided routing, and placement into extended grooves or channels, respectively. Multi-Tier indicates multiple task levels, task families, or difficulty settings; Multi-Source indicates demonstrations or trajectories from multiple sources under a shared schema. Deformable cells report the specified physics representation when available. ✓: supported. ✗: not supported. –: not applicable. Benchmarks are sorted by publication year.
Selectable transcription of PDF table (PDF reference numbering)
BenchmarkInsertionRoutingSeatingMulti-TierArticulated WireDeformable WireRL BaselineIL BaselineVLA BaselineSimulated RobotPhysical RobotMulti-Source
RLBench [12] (2020)✓✗✗✗✗✗✗✗✗✓✗✗
Zhou et al. [35] (2020)✓✗✗✗––✗✗✗✗✓✗
DeformableRavens [15] (2021)✗✗✓✓✓✓FEM✗✓✗✓✓✗
PlasticineLab [24] (2021)✗✓✗✓✗✓MPM✓✗✗✓✗✗
ReForm [17] (2021)✓✓✗✓✓✓AGX/LEM✓✗✗✓✗✗
SoftGym [14] (2021)✗✗✗✓✗✓PBD✓✗✗✓✓✗
NIST ATB [39] (2022)✓✓✗✓––✗✗✗✗✓✗
Jin et al. [5] (2022)✓✓✗✓––✗✓✗✗✓✗
Zhao et al. [33] (2022)✓✗✗✓––✓✓✗✗✓✓
Chang et al. [38] (2022)✓✓✗✓––✗✗✗✗✓✗
DaXBench [16] (2023)✗✗✗✓✗✓MPM/MS✓✓✗✓✓✗
Wilson et al. [3] (2023)✓✓✗✓––✗✗✗✗✓✗
Luo et al. [4] (2024)✓✓✗✓––✗✓✗✗✓✗
DexDLO [28] (2024)✗✗✗✓✓✓ART✓✗✗✓✗✗
Li and Choi [9] (2024)✓✗✗✓✗✓PBD✓✗✗✓✓✗
Zhang et al. [2] (2024)✓✓✗✓––✗✗✗✗✓✗
Chen et al. [26] (2025)✗✗✗✗✓✓DERH✗✗✗✓✓✗
DEFORM [29] (2025)✗✗✗✗✗✓DER✗✗✗✓✓✗
Yu et al. [36] (2025)✗✗✗✓✗✓DER✗✗✗✓✓✗
ManiSkill3 [13] (2025)✓✗✗✓✗✗✓✓✓✓✓✓
DEFT [30] (2025)✓✗✗✓✗✓B-DER✗✗✗✗✓✗
Govoni et al. [31] (2025)✗✗✗✗✗✓MSD✗✗✗✓✗✗
Li et al. [32] (2025)✓✓✗✓✗✓PBD✓✓✗✓✓✗
Li et al. [19] (2025)✓✓✗✓✗✓Particle✓✗✗✓✓✗
Kienle et al. [21] (2025)✓✗✗✓––✗✗✗✗✓✗
Tanureza et al. [20] (2025)✓✓✗✓––✗✗✗✗✓✗
Li et al. [37] (2025)✗✓✗✗✗✓MSD✗✗✗✓✗✗
Azulay et al. [40] (2025)✗✓✗✓––✗✗✗✗✓✗
MoDeSuite [25] (2026)✗✗✗✓✗✓FEM/PBD✓✓✗✓✓✗
DLO-Lab [18] (2026)✓✓✗✓✗✓Diff. DER✓✗✗✓✓✗
Cao et al. [34] (2026)✗✓✗✓––✗✗✗✗✓✗
WireCraft (Ours) (2026)✓✓✓✓✓✓FEM✓✓✓✓✓✓

Appendix B DLO Physics and Simulation Details

The appendix supplies simulator versions, model-specific parameters, coupling mechanisms and an isolated throughput experiment. These details define reproducible benchmark configurations without making the two models material-equivalent.

B.1 Simulation Backend

The default physics step is 1/120 s with decimation two, yielding 60 Hz control and eight-second RL episodes. The authors also checked Isaac Sim 5.1 and several GPU models; the detailed scaling curve specifically uses an RTX A4000.

B.2 DLO Models

Both example wires are 0.4 m long, but FEM uses 3 mm radius, density 800 and a forty-resolution mesh, while the articulated model uses twelve segments, 4 mm radius and density 2000. Joint stiffness/damping and elastic modulus are different solver-native controls and should not be compared numerically as equivalent material parameters.

Connector mass is 0.01 kg, connector friction is 2, gripper-pad friction 5 and default table friction 1. The complete table retains all contact, solver, collision and material settings; their physical accuracy has not been calibrated.

Complete original PDF block A2.T3
Table 3: Representative wire model parameters. Values reflect each model’s native representation and should be interpreted as solver-specific benchmark settings rather than physically equivalent material parameters. Articulated wire values correspond to the committed task-environment configuration.
Original selectable table
Setting FEM-based Deformable Wire Articulated Wire
Representation Isaac Sim deformable body volume (PhysX FEM) Rigid cylinder chain with D6 joints
Total length 0.4 m 0.4 m
Segments / resolution 40 segments; hexahedral resolution 40 12 rigid segments; ∼\sim0.033 m per segment
Radius / density 0.003 m; 800.0 0.004 m; 2000.0
Friction Dynamic 5.0; static does not support deformable body Static 1.0; dynamic 1.0
Material / joint parameters Young’s modulus 5.0×1075.0\times 10^{7}; Poisson’s ratio 0.49; vertex-velocity damping 3.0 Joint stiffness 5.0; damping 5.0; joint limit 0.5 rad
Solver / contact settings 32 position iterations; contact offset 0.001; rest offset 0.0 32 position iterations; 4 velocity iterations; max depenetration velocity 1.0
Collision / body settings Collision simplification enabled; sleep threshold 0.01 Self-collision enabled; max linear/angular velocity 5.0/60.0

B.3 DLO–Connector Interaction

FEM endpoints use deformable-vertex attachments with 0.015 m overlap offset; articulated endpoints use fixed joints welding the connector to the terminal segment. Figure 6 preserves the entire twenty-four-frame comparison, showing the consequences of the different coupling and body models.

Complete original PDF block A2.F6
Figure 6: DLO–Connector Physics Demonstration. Visualization of articulated and deformable wire concatenated with a DisplayPort connector across representative simulation frames.

B.4 Computational Scaling and Throughput

The throughput test removes robots, endpoint connectors and rendering, retaining DLO-only ground contact. Each configuration has five seconds of warmup and sixty seconds of measurement at 120 Hz physics. Articulated simulation succeeds through 8,192 environments and fails to set up at 16,384; FEM succeeds through 2,048 and fails at 4,096.

FEM is faster at the smallest counts, while the chain scales to much higher aggregate steps per second. The authors relate FEM failure to a 4,095-volume actor cap and potentially earlier fixed GPU-memory limits. This is configuration-specific physics throughput, not end-to-end rendered robot-policy speed or a fidelity-matched solver comparison.

Complete original PDF block A2.F7
Figure 7: Scaling behavior of the two WireCraft wire models. Both models were evaluated in matched headless DLO-only scenes with a 1/1201/120 s physics timestep, 55 s warmup, 6060 s measured rollout. Absolute throughput depends on discretization and solver settings; the comparison is intended to show scaling behavior of the selected benchmark configurations. The articulation model scales to 8192 successful environments, while the deformable solver-32 model succeeds through 2048 environments and fails at 4096 owing to PhysX deformable-volume limits.

Appendix C Task Design

C.1 Connector Insertion

Five connector assets range from axial-symmetric cylinders to cuboids, Ethernet-style, asymmetric DisplayPort and USB-A. Four appear in reported experiments; USB-A is released only as an additional asset. Geometry is fixed per task and socket clearance is a separate difficulty parameter.

C.2 Clip Routing

Routing curricula use one to three U-slot clips. Their longitudinal positions are fixed while lateral position and yaw vary; narrower openings and retaining lips add constraints. Longer routes may require re-grasping, but only the one-clip setting is quantitatively evaluated here.

C.3 Channel Seating

Five connected channel segments form a straight or bent path. Curved settings allow up to ±20° relative rotation per joint; the policy must position the wire body, not just move its endpoint near a goal. Reported baseline and hardware results use straight channels.

C.4 Randomization

The randomization table specifies connector drop height, roll/pitch/yaw and planar offsets, socket position, clip lateral/yaw ranges and optional channel bends. Insertion RL uses connector x/y offsets of ±5/±15 cm, socket y/z ranges of 28/15 cm and a 6–11 cm settling drop. Scripted and teleoperation collection use different planar ranges, so dataset-generation settings should not be mistaken for the RL benchmark distribution.

Complete original PDF block A3.T4
Table 5: Per-episode randomization for the WireCraft task families, grouped by the randomized component. The connector is attached to the DLO terminal and randomized before a short free-fall settling drop. Ranges are reported for the RL training and evaluation configuration.
Original selectable table
Component Parameter Default range / description
Connector Spawn height 66–1111 cm above the table, then free-fall settling
Roll / pitch Uniform [−18∘,18∘][-18^{\circ},18^{\circ}]
Yaw Uniform [−36∘,36∘][-36^{\circ},36^{\circ}]
XX offset Uniform ±5\pm 5 cm
YY offset Uniform ±15\pm 15 cm
Socket XX position Fixed along the insertion axis
YY position Randomized within a 2828 cm range
ZZ position Randomized within a 1515 cm range
Clips XX position Fixed along the routing line
YY position Uniform within ±6\pm 6 cm of the nominal lane
Yaw Uniform [−15∘,15∘][-15^{\circ},15^{\circ}]
Channels Position Fixed
Shape 55 segments; each adds a signed bend of up to ±20∘\pm 20^{\circ} to the running heading

C.5 Control Modes

Controllers expose direct joint targets, Cartesian pose deltas or Cartesian translation with fixed orientation. Differential inverse kinematics converts Cartesian commands into the common joint-target interface, with a gripper command in every mode. Fixed orientation suits seating; insertion and routing require orientation control.

Appendix D Scripted Policy

A vectorized finite-state machine advances through approach, grasp and task phases after pose tolerances and dwell conditions are met. It clips pose errors into Cartesian delta commands and snapshots relevant frames at grasp time, preventing wire motion from continually moving the intended placement reference.

Complete original PDF block A4.T5
Table 6: Comparison of the three scripted policies. All share the vectorized finite-state-machine framework and a Cartesian delta-pose action; they differ in their grasp target, control dimensionality, core sub-skill, and the perturbations injected for demonstration diversity.
Original selectable table
Insertion Clip routing Channel seating
Grasp target Connector tail DLO body node near the clip Connector
Action 7-DoF (pose + gripper) 7-DoF (pose + gripper) 7-DoF with fixed end-effector orientation
Core sub-skill Align and drive into the socket Over-stretch, then push the DLO into the slot Lift and slowly place into the channel
Reference frame Socket pose Clip pose with a frozen anchor-to-clip direction Channel center and length
Success Connector seated in the socket DLO node inside the clip slot DLO body laid within the channel over 80%
Injected perturbation Pick-and-drop (30%30\%) and failure retry Pre-stretch with a frozen target direction Delayed grasp and slowed placement

D.1 Insertion

Insertion grasps the connector tail, lifts, aligns the held connector to the socket, advances and releases. A 30% pick-and-drop perturbation creates recovery examples, and unsuccessful seating can trigger up to two approach retries. Pitch/grasp-offset randomization and local speed reductions diversify the trajectories.

D.2 Clip Routing

Routing grasps the wire body nearest the clip, with gripper orientation aligned to its local tangent. It lifts and rotates, over-stretches along a frozen anchor-to-clip direction, pulls back to release tension, then descends past the retaining lip. Grasping the connector would fail geometrically because it is wider than the slot.

D.3 Channel Seating

Seating grasps the connector after a settling delay, lifts gently and follows the channel at reduced speed with fixed orientation. Fine action scaling and gradual path descent aim to avoid snagging; following a path permits both straight and curved task definitions.

D.4 Scripted Demonstrations

Figure 8 keeps all twelve snapshots of the three scripted task rollouts, including grasp, transport and fixture engagement.

Complete original PDF block A4.F8
Figure 8: Scripted demonstration rollouts. Each row shows a scripted policy executing one task family across representative phases, progressing from the initial state through grasp and manipulation to task completion.

Appendix E RL Experiment Detail

E.1 State-based RL

Privileged insertion PPO and SACfD share state, action and shaped reward, with algorithm-specific normalization and networks. Routing and seating reuse PPO hyperparameters with task-specific observations/rewards. Reported RL means and standard deviations use three seeds.

Observation.

The forty-two-dimensional insertion state includes joint/gripper positions and velocities, end-effector pose, two fingertip contacts, connector/socket/seated-target geometry and gate/error scalars. This exposes goal-relative information unavailable to an RGB-only actor.

Action and control.

A six-dimensional Cartesian delta plus gripper command is clipped to [−1,1] and scaled by 0.01 before differential inverse kinematics. The RL setting runs at 60 Hz for eight-second episodes; imitation-learning evaluation later specifies its own 800-step horizon.

Reward.

The common reward shapes reaching, grasping, bringing, alignment and orientation, with a large insertion-depth progress term and seated bonus. Table/gripper and action penalties discourage undesirable motion. Success under this detailed privileged reward does not measure how easily the task can be learned from sparse visual feedback.

Networks.

PPO uses 256/128/64-unit ELU actor and critic networks. SACfD uses 256/256-unit actor and twin critics, about 150,000 scripted transitions, a 50% demonstration replay ratio and an annealed behavior-cloning weight. The two full hyperparameter tables retain the exact schedules and budgets.

Complete original PDF block A5.T6
Table 7: State-based PPO hyperparameters. Vision PPO (Sec. E.2) inherits these except where noted. num_envs is set at launch with no committed default.
Original selectable table
Parameter Value
Actor / critic MLP [256,128,64][256,128,64], ELU
Clip range ϵ\epsilon 0.20.2
Entropy coef. 0.010.01
Value-loss coef. 1.01.0
GAE λ\lambda 0.950.95
Discount γ\gamma 0.990.99
Learning rate 5×10−45\times 10^{-4}, adaptive (KL∗=0.01\mathrm{KL}^{*}{=}0.01)
Steps per env (rollout) 128128
Minibatches / epochs 88 / 55
Max grad norm 1.01.0
Max iterations 2000020000
Obs. normalization empirical (on)
Complete original PDF block A5.T7
Table 8: SACfD hyperparameters on connector insertion tasks.
Original selectable table
Parameter Value
Actor / critic MLP [256,256][256,256], ELU
Discount γ\gamma 0.990.99
Target smoothing τ\tau 0.0050.005
Actor / critic LR 3×10−43\times 10^{-4} / 3×10−43\times 10^{-4}
Batch size 256256
Gradient steps / env step 11
Replay capacity 8000×Nenv8000\times N_{\text{env}}
Warmup (learning starts) 50005000
Entropy temp. α\alpha learned, target −7-7, α0=0.1\alpha_{0}{=}0.1
Reward shaping scale 0.010.01
State preproc. RunningStandardScaler
Total timesteps 10610^{6}
Demonstration augmentation (SACfD)
Demo set ∼\sim150k transitions (scripted)
Demo replay ratio 0.50.5
BC weight (anneal) →0.52.5\!\to\!0.5 over [80​k,250​k][80\text{k},250\text{k}] steps

E.2 Vision PPO

Design rationale.

Vision PPO first behavior-clones a visual actor from state-policy rollouts, then applies online PPO. This uses privileged teachers and pretrained visual representations; it is not vision-only learning from scratch.

Architecture.

The actor receives two 224×224 RGB images and fourteen proprioceptive values; the critic retains the forty-two-dimensional privileged state. A DINOv2 backbone with eight pooling queries per camera supplies the actor features. During PPO both backbone and attention pool are frozen; only the output projection, action noise and critic train. The critic starts from the converged state policy.

Hyperparameters.

Visual fine-tuning uses sixty-four environments and six hundred iterations, with clip 0.1, target KL 0.005, learning rate 10⁻⁴ and small initial action noise. It removes entropy bonuses and observation normalization and reduces the value-loss weight. Thus its budget and trainable modules differ markedly from privileged PPO.

E.3 RL Rollout Demonstrations

Figure 9 retains all twelve State-PPO snapshots, including a clip-routing error and recovery. These illustrate closed-loop trajectory characteristics without quantifying a general recovery rate.

Complete original PDF block A5.F9
Figure 9: RL rollout demonstrations. Each row shows a trained State PPO policy rolled out on one task family across representative phases. The clip-routing row additionally illustrates an error and subsequent recovery.

Appendix F Connector-Insertion IL/VLA Experimental Details

F.1 Dataset

The four insertion IL/VLA baselines share one thousand scripted trajectories, two RGB views and seven-dimensional UR5 joint/gripper state and actions. Sharing this dataset does not equalize pretrained knowledge, architecture or training exposure.

F.2 Policy Training Configurations

ACT trains for 100k steps, Diffusion Policy 200k, DiT VLA 300k and π₀.₅ 50k, with different batch sizes, horizons and executed action chunks. DiT uses frozen CLIP image/text conditioning; π₀.₅ freezes the vision tower while adapting the action expert and language adapter. The full configuration table is retained.

Complete original PDF block A6.T8
Table 9: Training configurations for the IL/VLA connector-insertion baselines. All policies use the same dataset and action/state encoding.
Original selectable table
Policy Model / conditioning Action prediction Training
ACT ResNet-18 visual encoder with transformer action-chunking policy Chunk size 100; 50 executed actions per inference call 100k gradient steps; batch size 64; learning rate 10−510^{-5}
Diffusion Policy ResNet-18 visual encoders with a temporal U-Net action denoiser Horizon 16; 2 observation steps; 8 action steps 200k gradient steps; batch size 64; learning rate 10−410^{-4}
Diffusion Transformer VLA Frozen CLIP ViT-B/16 image and text conditioning with a DiT-style action denoiser Horizon 32; 24 action steps 300k gradient steps; batch size 32; learning rate 2×10−52\times 10^{-5}
π0.5\pi_{0.5} Initialized from lerobot/pi05_base; vision tower frozen; action expert and language adapter trainable 25 action steps 50k gradient steps; batch size 32; learning rate 10−510^{-5}

F.3 Checkpoint Selection

Three checkpoints per policy are retained using reach success on evaluation rollouts, rather than an independent validation split. The displayed best-observed results are optimistic under this selection procedure. Insertion success comes from the same selected checkpoint and is not independently optimized, but its correlation with reach does not make selection independent of test performance.

F.4 Evaluation Protocol

Each retained checkpoint receives four seeds with sixteen rollouts each, recording reach and insertion from the same trajectory over an 800-simulator-step horizon. All twelve checkpoint rows are preserved: ACT improves substantially by 100k, while π₀.₅ has zero insertion success for all three retained checkpoints. These four evaluation seeds differ from the three-seed RL/routing/seating protocols.

Complete original PDF block A6.T9
Table 10: Available and retained checkpoints for the connector-insertion IL/VLA baselines. Retained checkpoints are the top three checkpoints per policy according to online reach success during training.
Original selectable table
Policy Number of available checkpoints Retained checkpoints
ACT 20 25k, 40k, 100k
Diffusion Policy 20 140k, 190k, 200k
Diffusion Transformer VLA 23 110k, 180k, 200k
π0.5\pi_{0.5} 5 10k, 20k, 30k
Complete original PDF block A6.T10
Table 11: Connector-insertion results for the retained checkpoints selected by reach success. Each entry reports mean ±\pm standard deviation over four seeds, with 16 rollouts per seed.
Original selectable table
Policy Checkpoint SRreach\mathrm{SR}_{\mathrm{reach}} (%) SRinsert\mathrm{SR}_{\mathrm{insert}} (%)
ACT 25k 43.75±11.6943.75\pm 11.69 3.13±3.133.13\pm 3.13
ACT 40k 54.69±17.8854.69\pm 17.88 9.38±6.999.38\pm 6.99
ACT 100k 60.94±11.16\mathbf{60.94\pm 11.16} 29.69±9.24\mathbf{29.69\pm 9.24}
Diffusion Policy 140k 29.69±18.4229.69\pm 18.42 17.19±13.5317.19\pm 13.53
Diffusion Policy 190k 35.94±9.2435.94\pm 9.24 17.19±6.8117.19\pm 6.81
Diffusion Policy 200k 37.50±9.88\mathbf{37.50\pm 9.88} 20.31±6.81\mathbf{20.31\pm 6.81}
Diffusion Transformer VLA 110k 9.38±6.999.38\pm 6.99 3.13±3.133.13\pm 3.13
Diffusion Transformer VLA 180k 4.69±5.184.69\pm 5.18 1.56±2.711.56\pm 2.71
Diffusion Transformer VLA 200k 10.94±2.71\mathbf{10.94\pm 2.71} 3.13±3.13\mathbf{3.13\pm 3.13}
π0.5\pi_{0.5} 10k 0.00±0.000.00\pm 0.00 0.00±0.000.00\pm 0.00
π0.5\pi_{0.5} 20k 6.25±4.42\mathbf{6.25\pm 4.42} 0.00±0.00\mathbf{0.00\pm 0.00}
π0.5\pi_{0.5} 30k 1.56±2.711.56\pm 2.71 0.00±0.000.00\pm 0.00

Appendix G Clip Routing and Channel Seating Experimental Details

Datasets.

Routing uses 1,600 scripted trajectories and straight seating one thousand, shared between ACT and DiT VLA within each task.

Training.

ACT trains for 100k steps with batch 64 and executes fifty of each predicted hundred-action chunk. DiT trains for 300k with batch 32, two observations, horizon thirty-two and twenty-four executed actions. All four training runs use seed 1000; evaluation-seed variation does not measure training-seed variability.

Checkpoint selection and evaluation.

These tasks evaluate the final training checkpoint over three evaluation seeds. Their selection protocol is therefore different from the insertion study’s best reach-selected checkpoints.

Appendix H Real-World Validation

The hardware study examines whether matched simulation data supplement real demonstrations. It tests ACT on three physical tasks using four real-containing mixtures after poor preliminary zero-shot simulation-only results.

H.1 Setup

Physical setup.

UR5 uses a Robotiq 2F-85 gripper and two RealSense D405 RGB views, with a 6 mm HDMI cable for insertion/routing and 10 mm cord for seating. ACT runs at 30 Hz using RGB and proprioception; neither depth nor the mounted force/torque sensor is used. Simulation matches its tool offset, cameras and task-board appearance.

Task initialization.

The insertion connector starts within 3×3 cm and the socket is fixed. Routing places one clip 2–5 cm from the wire; seating fixes one endpoint and starts the other 2–5 cm from the channel. These restricted setups limit direct comparison to the broader simulation randomization.

Bridging the sim-to-real visual gap.

Only simulated training frames receive brightness/contrast factors in [0.85,1.15] and Gaussian noise with standard deviation up to 0.015 on normalized pixels. Real training images and evaluation inputs are unaugmented. Figure 10 preserves all twelve domain/view/task examples.

Complete original PDF block A8.F10
Figure 10: Real vs. noise-augmented simulation observations across three tasks. Rows show connector insertion, clip routing, and channel seating; columns show the side and wrist views from real demonstrations and scripted simulation trajectories. Brightness, contrast, and Gaussian-noise augmentation are applied only to the simulated frames using the training preset.

H.2 Protocol

Data sources.

Each task has fifty real episodes, except routing with fifty-one, plus two hundred scripted and two hundred RL episodes. Every tenth episode is held out, giving five real validation episodes per task. Full frame counts and source-specific train/validation splits are preserved in Table 12.

Complete original PDF block A8.T11
Table 12: Trajectory data for the three hardware tasks by source. Episodes and frames report total dataset sizes; train and validation columns report episode counts. All sources are recorded at 3030 FPS under the shared schema.
Original selectable table
Task Source Episodes Frames Train Validation
Insertion Real-world (R) 5050 30,01830{,}018 4545 55
Scripted (S) 200200 121,890121{,}890 180180 2020
RL rollout (P) 200200 27,39427{,}394 180180 2020
1-clip routing Real-world (R) 5151 30,63430{,}634 4646 55
Scripted (S) 200200 125,304125{,}304 180180 2020
RL rollout (P) 200200 53,22653{,}226 180180 2020
Straight-channel seating Real-world (R) 5050 30,03330{,}033 4545 55
Scripted (S) 200200 114,954114{,}954 180180 2020
RL rollout (P) 200200 56,51156{,}511 180180 2020

Training.

Every mixture trains ACT for 80k steps, batch sixteen, learning rate 10⁻⁵ and seed 1000. ImageNet-initialized ResNet-18 encoders feed hundred-action predictions, with fifty actions executed per inference. A common update budget does not make the amount or distribution of training data equal.

Checkpoint selection.

Checkpoints saved every 5k steps are selected by masked normalized L1 action loss on only the five held-out real episodes, including mixed-data policies. Selected steps vary by mixture and task. Real-only normalization statistics include those held-out episodes, so the preprocessing split is not strictly independent; a small action-loss validation set also does not directly optimize physical success.

H.3 Results and Analysis

The full failure montage distinguishes missed grasps, unsuitable connector alignment, clip misses and incomplete body placement. These observations explain candidate bottlenecks but do not provide independent causal ablations.

Connector insertion.

Insertion can fail during pickup or after reaching the socket because the connector’s pose within the grasp prevents seating. Transport success and an insertion-compatible grasp are different requirements.

Clip routing.

Routing may release beside the clip or leave the wire resting on its lip. Reaching the fixture is insufficient when the wire is not actually seated.

Channel seating.

Seating failures include missed endpoint grasps and portions of the wire remaining outside the channel. Source trajectories also differ: human episodes contain corrections, scripts are slower and varied, and collected RL rollouts run at half their original policy speed.

Complete original PDF block A8.T12
Table 13: Characteristics of the three trajectory sources used for hardware policy training.
Original selectable table
Source Trajectory characteristics
R Human demonstrations contain pauses and repeated fine adjustments during alignment. Their limited diversity can restrict generalization.
S Scripted trajectories use action noise during generation to broaden motion coverage. Execution is relatively slow and regular, providing varied examples of controlled task completion.
P Task-optimized RL rollouts produce fast, aggressive motions. For this dataset collection, execution speed was reduced to 50% of the original policy speed.
Complete original PDF block A8.F11
Figure 11: Representative hardware failures in insertion (left), clip routing (middle), and channel seating (right). Panels show (a,c) missed grasps, (b) wire misplacement beside the clip, (d) reaching without connector seating, (e) wire resting on the clip rim, and (f) wire placement outside the channel.

Limitations.

Twenty trials imply five-percentage-point success increments: 65% versus 60% routing is only one trial. A single robot, narrow initialization and changing data sizes preclude a definitive general ranking or an isolated source-diversity effect.

Appendix I Teleoperation Data Collection

SpaceMouse, Meta Quest wrist/pinch control or combined desktop-control/VR-viewing modes produce Cartesian deltas, resolved by inverse kinematics to joint targets. Recorded actions are those applied joint targets, not raw device inputs. The common schema also stores both cameras, robot state, end-effector pose, timestamps, episode index and task label.

The eight-stage demonstration preserves repeated grasp/alignment attempts caused by limited visual cues and absent haptic feedback. Simulation teleoperation tools are available, but systematic collected-data filtering and policy training with those trajectories remain future work.

Equation
acart=[Δ​x,Δ​y,Δ​z,Δ​rx,Δ​ry,Δ​rz,g],a_{\mathrm{cart}}=[\Delta x,\Delta y,\Delta z,\Delta r_{x},\Delta r_{y},\Delta r_{z},g],
Complete original PDF block A9.F12
Figure 12: A teleoperated connector-insertion trajectory in simulation. A human operator drives the UR5 end-effector through eight representative stages: (1) initialization with the connector resting on the table; (2) approaching the connector; (3) aligning the gripper above it; (4) grasping the connector, typically after several attempts, since the operator must re-position and re-close the gripper to secure a firm grasp; (5) lifting the connector clear of the surface; (6) moving toward the socket while pulling the trailing wire taut to reduce its disturbance during insertion; (7) aligning the connector with the socket opening; and (8) seating it, often requiring repeated insertion attempts to correct small misalignments. Because the operator has no haptic feedback and only limited visual cues for fine alignment, both grasping and insertion rely on trial-and-error retries, making teleoperated episodes substantially longer than scripted or policy rollouts.
Concept demonstration

Open image ↗