Overview
Play2Perfect first learns general grasping and in-hand pose control in simulation, then specializes that prior to contact-rich assembly with sparse CAD-derived goal rewards.
![[Uncaptioned image]](https://storage.literature.yihao.one/media/sha256/2b/2be7aa07e9fb6fb5ddcfb39501890f1b9a7271c135043a94142d64c0c75f70af.webp)
At a glance
A goal-conditioned dexterous policy is pretrained by manipulating diverse primitive objects through random poses, then finetuned for insertion, beam assembly, or screwing. Ablations favor object diversity, coupled translation/rotation, varied trajectories, and precise play goals. Hardware insertion succeeds in six of ten trials at 0.5-millimetre nominal clearance; complete beam assembly succeeds in twelve of twenty and complete insertion-plus-screwing in five of ten. The reported thirty-three-fold learning speedup concerns downstream adaptation on a simplified comparison task and excludes the seven-day shared pretraining cost.
1 Introduction
Sparse assembly success is hard to discover from a random arm-and-hand policy because grasping, reorientation, transport, alignment, and contact must all work first. Play pretraining supplies reusable manipulation behaviors before assembly-specific optimization. The central claim is that useful play teaches finger-driven in-hand pose control, not merely moving a rigidly grasped object with the arm. The experiments study this prior and its specialization rather than a complete autonomous assembly planner.
2 Related Work
Dexterous imitation learning faces teleoperation and contact-feedback difficulties, while reinforcement learning often relies on dense rewards or specialized demonstrations. Previous play controllers mostly target free-space behavior or direct transfer. Play2Perfect uses play as an initialization for further task-specific learning, with CAD-defined sparse contact goals. Assembly environments and CAD structure are still supplied, so the approach reduces demonstration and dense-reward engineering without removing all task specification.
3 Play2Perfect
The same arm/hand policy is first trained on procedural objects and random free-space goals, then adapted in simulation to assembly environments and goal sequences derived from CAD. The final controller consumes object state and a goal pose rather than raw visual or tactile observations.
3.1 Dexterous Play Pretraining
The policy observes robot proprioception, current/target object poses, and bounding-box geometry. Procedural primitive combinations vary dimensions, density, centre of mass, and inertia. A play episode begins with grasp/lift and continues through nearby positions with substantial orientation changes, encouraging repeated in-hand reorientation.
Pretraining is shaped reinforcement learning: smoothness, grasp/lift, and pose-progress rewards accompany a large sparse goal-success bonus. The default one-centimetre keypoint-distance tolerance encourages fine pose control, while online random goal sequences broaden transitions. The later claim of sparse assembly rewards should not be read as sparse-only training throughout the entire pipeline.
3.2 RL Finetuning on Assembly Tasks
An assembly-by-disassembly procedure reverses feasible part removals to define assembly steps. Each manipulated part has a CAD-derived relative transform to the assembled fixture. Sparse intermediate goals include a pre-contact insertion pose or successive ninety-degree screw-thread poses, followed by the final assembled pose. Finetuning retains smoothness regularization and sparse goal/retraction bonuses but removes play’s dense grasp/lift/progress terms. Thus supervision includes intermediate CAD-derived goals, not only one terminal reward at the end of the complete multi-part task.
3.3 Training Details and Sim-to-Real Transfer
Both stages use Split and Aggregate Policy Gradients with domain randomization for delays, pose errors, and dynamics. Deployment uses FoundationPose and the known CAD geometry. The controller runs at sixty hertz and part tracking at thirty; the detailed appendix specifies that fixture pose is estimated once at rollout start, while the part is tracked online.
4 Experiments
A twenty-two-degree-of-freedom Sharpa hand on a seven-joint KUKA arm is evaluated on tight insertion, two-step beam assembly, and screw-leg manipulation. Beam parts from Fabrica are printed at triple scale, and the FurnitureBench leg is made three times longer to improve grasping/tracking visibility. Four learned skills count beam’s two steps separately. Simulation evaluates five hundred randomized rollouts; hardware uses ten or twenty trials with fixed fixtures and varied starting parts. Completion time is reported over successful trials only.
4.1 Can Dense Rewards Overcome the Need for Play ?
On the main tasks, the pretrained policy reaches high success after roughly two to five hours of downstream learning, while sparse- and dense-reward scratch policies produce no successes after twenty-four hours. A simplified propped-up insertion initialization lets dense scratch learning succeed after over one hundred hours, versus about four hours for Play2Perfect; the authors report a thirty-three-fold downstream speedup.
That comparison does not include the common play prior’s seven days of pretraining. The scratch policy also learns a brittle balancing strategy, dropping to roughly twenty percent under a ten-newton perturbation and zero under stronger forces, whereas the pretrained strategy remains above seventy-five percent at the largest tested perturbations. These results isolate adaptation and strategy robustness within the tested initialization and force range.
4.2 Which Design Choices in Play Pretraining Matter Most for Downstream Assembly?
Ablations average four assembly skills over three seeds. Increasing procedural objects from ten through one hundred to one thousand improves stability and final success. Translation-only play lacks the orientation skills needed for assembly; rotation-only transfers substantially better but plateaus about ten points below coupled six-dimensional control. Online random trajectories outperform fixed banks of ten or one hundred. A ten-centimetre success tolerance transfers poorly, and five centimetres learns more slowly and plateaus near sixty percent. These are controlled changes within the paper’s pretraining recipe, not proof that object count alone captures all relevant diversity.
4.3 Is RL Finetuning Necessary for Precise, Contact-Rich Assembly?
Clearance is defined as the full difference between hole and peg cross-sectional dimensions: a thirty-by-twenty-millimetre peg in a thirty-one-by-twenty-one hole has one-millimetre nominal clearance. Training spans five to 0.5 millimetres; simulation tests extend down to 0.2, where finetuned success is eighty percent. Play-only reaches about seventy-five percent at forty millimetres but nearly zero by four. Hardware finetuning improves ten-millimetre, two-millimetre, and 0.5-millimetre results from 60%, 20%, and 0% to 100%, 90%, and 60%. The one-centimetre pose-goal tolerance and submillimetre geometric clearance are different quantities; contacts constrain the actual insertion.
4.4 Can Play2Perfect Transfer to the Real World?
Hardware insertion succeeds in 10/10, 9/10, and 6/10 trials as clearance tightens. Successful-run mean time increases from 6.8±1.5 seconds to 11.1±5.1 seconds. Beam step one succeeds 16/20 times; step two succeeds 12/16 conditional on step one, giving 12/20 end-to-end. Screw insertion succeeds 7/10, and subsequent screwing succeeds 5/7, giving 5/10 complete runs. These conditional rates must not be multiplied or averaged as if they came from independent fresh-start trials.
Reported successful full-task times include approach, grasp, transport, and contact, with beam at 21.0±8.9 seconds and insertion-plus-screwing at 15.6±2.9. There is no matched hardware timing baseline establishing a general speed advantage over grippers. Among sixty recorded real trials, eighteen failures include six timeouts, five insertion-phase drops, three reorientation drops, and four catastrophic tracking losses; subtler pose errors may also contribute.
| Tight-Insertion | Assemble-Beam | Screw-Leg | |||||
| 10mm | 2mm | 0.5mm | Step 1 | Step 2* | Insert | Screw* | |
| Success Rate | 10/10 | 9/10 | 6/10 | 16/20 | 12/16 | 7/10 | 5/7 |
| Completion Time | 6.8 1.5 s | 9.4 1.9 s | 11.1 5.1 s | 21.0 8.9 s | 15.6 2.9 s | ||
4.5 Does Play2Perfect Transfer to Everyday Insertion Tasks?
The same pretrained checkpoint and finetuning recipe reaches ninety percent simulated success for iPhone-plug insertion after 8.4 hours and fork-to-rack insertion after 2.8 hours. These results extend the tested object/task geometries in simulation; they are not additional physical demonstrations of small-object perception or everyday household reliability.
5 Discussion and Limitations
Play pretraining enables efficient specialization to precise contact behaviors, but the system comprises short assembly skills with externally specified task sequencing, active parts, and goals. Policies are finetuned per task or benchmark family. Pose tracking can fail under hand occlusion or rapid motion, and the policy lacks direct observation of fixture/surrounding geometry beyond its goal representation. Broader sequencing, memory, multi-task learning, and visual/tactile input remain future directions.
Acknowledgements
Support includes Stanford Human-Centered Artificial Intelligence, an ONR Young Investigator Award, NSF grants, NSERC funding, and research/technical collaboration with Sharpa. The original acknowledgment retains named contributors and grant identifiers.
Appendix A Robustness to Noisy Pose Tracking and Failure Analysis
Simulation noise sweeps show tolerance beyond the training regime, with Screw-Leg the most sensitive skill. The appendix prose summarizes one-centimetre/five-degree pose noise, while the finetuning table distinguishes current-part noise from smaller goal-pose noise of two millimetres/one degree. The retained failure chart separates easily identified catastrophic tracking loss from other failures whose possible small pose errors cannot be isolated without ground truth.
Appendix B End-to-End Two-Step Beam Assembly
Twenty physical beam trials execute step two directly from successful step-one outcomes without resetting the intermediate assembly. Sixteen reach the second step and twelve finish both, yielding sixty percent overall. The four failed first steps and four failed second steps are part of that denominator.
Appendix C Additional Ablation Results
Per-task curves repeat object-count, objective, trajectory-bank, and goal-tolerance ablations on insertion, both beam steps, and screwing. They broadly support the averaged finding that precise orientation-capable play transfers better than translation-only manipulation. All curves remain available to assess differences in learning speed, saturation, and task-specific variation rather than relying solely on the aggregate trend.
Tight-Insertion
Assemble-Beam Step 1
Assemble-Beam Step 2
Screw-Leg
Appendix D Pose Tracking Failures with Smaller Objects
Original-size parts are heavily occluded by the hand, making real pose tracking unreliable. The figures show this limitation and motivate larger beam parts and a longer leg. Small-object simulation success does not remove the corresponding hardware perception bottleneck.
Appendix E Real-World Experiment Additional Analysis
Qualitative rollouts show tilted approaches, local contact search, corrective in-hand motion, and regrasping after recoverable drops without a separate scripted recovery controller. Recovery depends on the object remaining in the workspace and continued usable pose estimates; it is not guaranteed for arbitrary failures.
Fast motion and occlusion can break tracking. The nearly ninety-degree rotational symmetry of the rectangular leg causes orientation ambiguity, so colored tape differentiates its faces. Fixtures are immovable in the main training simulation but taped to a foam tabletop on hardware, where contact can move or deform them. That mismatch matters particularly because the fixture goal estimate is fixed at rollout start. Most control failures occur during final contact interactions rather than ordinary grasp acquisition.
Appendix F Unified vs. Task-Specific Policies
A unified policy co-trained across the four assembly skills achieves success comparable to separate policies in the reported experiment. This is a finite multi-task result within the studied family, not an unconditioned universal assembly controller.
Appendix G Training with Movable Fixtures
Additional policies trained and evaluated with movable fixtures remain close to fixed-fixture performance for tight insertion and beam step one. Beam step two and screwing are more vulnerable to fixture displacement. This study changes training as well as evaluation and should not be read as evidence that the main fixed-fixture policy is equally robust without adaptation.
Appendix H Simulation and Computational Resources
Training uses one RTX A6000 with physics at 120 hertz and control at sixty. Play runs seven days with 24,576 parallel environments; each assembly finetuning run is budgeted one day with 12,288 environments, though useful success emerges earlier in learning curves. Contact-rich geometry increases memory costs. Within each wall-clock comparison figure, methods share GPU type, environment count, hyperparameters, and training budget; the pretraining cost remains separate from downstream speedup.
Appendix I Policy Architecture and RL Algorithm
The actor uses a 1,024-unit recurrent layer followed by a multilayer network to output twenty-nine joint commands. An asymmetric critic additionally receives noise-free, undelayed state, velocities, rewards, and progress features unavailable to the deployed actor. SAPG supplies population-based exploration; architecture and optimization settings are shared between play and finetuning. The full hyperparameter table remains available.
| Parameter | Value |
| Actor network | LSTM[1024] + MLP[1024, 1024, 512, 512] |
| Critic network | MLP[1024, 1024, 512, 512] |
| Learning rate | |
| Minibatch size | 98,304 |
| SAPG block size | 4,096 |
| Entropy bonus scale | 0.002 |
| Discount factor | 0.99 |
| GAE parameter | 0.95 |
| PPO clip range | 0.1 |
Appendix J Play Pretraining Details
Procedural objects combine a graspable primitive with a smaller component near one end, varying dimensions and independent densities to change shape and inertia. Episodes randomize arm/hand initialization and object placement, then sample reachable initial goals and later goals within ten centimetres and ninety degrees. A ten-second horizon and termination conditions bound play, including table contact force above one hundred newtons.
Four object-frame keypoints jointly encode translation and rotation. Observations use dimensions of the graspable primary component, while reward distance uses fixed 14-by-3-by-3-centimetre keypoint geometry to keep its translation/rotation tradeoff consistent. The 140-dimensional observation contains proprioception, palm-relative geometry/current pose, goal displacement, and object dimensions. Arm outputs are incremental joint targets, hand outputs absolute targets; limits and exponential smoothing are applied.
Pretraining combines velocity regularization, fingertip approach/lift shaping, best-so-far pose-progress rewards, and a sparse goal bonus. A lift flag activates after ten centimetres; goal and lift bonuses are 1,000 and 300. Pose/action/proprioceptive delays, geometry/table variation, and force/torque perturbations encourage transfer. All original formulas and randomization values are retained.
Equations 1–9
| (1) |
| (2) |
| (3) |
| (4) |
| (5) |
| (6) | ||||
| (7) | ||||
| (8) |
| (9) |
| Parameter | Randomization |
| Current/goal pose noise (translation) | cm |
| Current/goal pose noise (rotation) | |
| Object-pose delay | – control steps |
| Action and proprioception delay | – control steps |
| Joint-velocity observation noise | rad/s |
| Object-dimension scale | – |
| Table-height variation | cm |
| External force perturbation | N |
| External torque perturbation | N m |
Appendix K Pretraining Ablation Implementations
Ablations vary one pretraining factor while keeping architecture, reward coefficients, randomization, and downstream finetuning fixed. Object sets contain ten, one hundred, or one thousand procedural samples. Translation-only ignores goal orientation; rotation-only first grasps/lifts to a pose and then changes orientation at fixed position. Fixed trajectory banks use the same sampling bounds as online trajectories. Goal thresholds are one, five, or ten centimetres, with the rest of the objective unchanged.
Appendix L Assembly Finetuning Details
Assembly simulation uses convex decomposition for most geometry and resolution-256 signed distance fields at narrow mating interfaces, preserving contact shape without applying the most expensive representation everywhere. Parts begin at randomized planar positions and orientations before settling; fixtures also have randomized placements. CAD relative transforms make goals fixture-relative. Pre-insertion poses or ninety-degree thread steps define the sparse goal sequence.
Finetuning removes dense reaching/grasping/lifting/progress rewards. It retains joint-velocity regularization, a 1,000-point bonus on reaching each one-centimetre-tolerance goal, and a 1,000-point terminal retraction bonus when the assembled part remains at its goal and the palm retreats more than twenty centimetres. This explicitly rewards release/retraction as well as assembly.
Current-part observations receive one-centimetre/five-degree noise; goal-pose observations receive two-millimetre/one-degree noise. Fixture yaw varies within ten degrees, with pose delay up to ten control steps, action/proprioception delay up to three, dimension/table variation, and external forces/torques. These structured randomizations approximate specific transfer errors rather than covering every possible contact or perception failure.
Equations 10–14
| (10) |
| (11) |
| (12) |
| (13) |
| (14) |
| Parameter | Randomization |
| Current part-pose noise (translation) | cm |
| Current part-pose noise (rotation) | |
| Goal-pose noise (translation) | mm |
| Goal-pose noise (rotation) | |
| Fixture yaw | |
| Object-pose delay | – control steps |
| Action and proprioception delay | – control steps |
| Joint-velocity observation noise | rad/s |
| Part-dimension noise scale | – |
| Table-height variation | cm |
| External force perturbation | N |
| External torque perturbation | N m |
Appendix M Inference Time Pipeline
At rollout start, CAD geometry supplies the part model, grasp bounding box, and fixture-relative goal sequence; the fixture pose is estimated once. FoundationPose updates the part estimate at thirty hertz, and the sixty-hertz controller reuses the newest pose between updates. A goal advances when the keypoint pose error falls within tolerance. Joint targets go to the existing low-level controllers. Learned contact search, in-hand rotation, and regrasping operate inside this externally supplied goal-sequencing interface, without an additional scripted insertion, screw, or recovery controller.