At a glance
RAPiD learns deformable-object manipulation in simulation, first using privileged particle histories and then replacing them with learned embeddings inferred from depth images, joint angles and actions. The frozen policy adapts its conditioning during execution without updating network weights. On a 22-DoF TIAGo mobile manipulator, it succeeds in 17/20 insertion trials and 16/20 covering trials on real objects, for 82.5% overall.
I Introduction
Different materials call for different motions: placing a stiff hat over a bowl differs from flinging or sweeping soft cloth. RAPiD extends Rapid Motor Adaptation by using recent simulated particle positions to teach a representation of changing shape, in addition to object mass and position. A second stage learns to infer the useful representation from onboard sensing.
The result is a learned policy with a changing latent context, rather than an explicit online fit of physical stiffness or a particle simulator running on the robot. This edition follows all eight pages of v1, dated 18 March 2026. The PDF’s entire introduction and Figures 1–3 were absent from the HTML; they are restored here, along with corrected figure/section numbering, all six complete figures, the full results table and 52 references.

II Related Work
III RAPiD: R apid A daptation of P art i cle D ynamics
Training has two stages. First, jointly learn a visuomotor policy and privileged shape/dynamics encoders in simulation. Second, train observation-based modules to imitate their latent outputs and fine-tune the policy on those predicted embeddings. Deployment discards the privileged encoders and freezes all learned weights.
The source formulates control as a finite-horizon Markov decision process with simulator state, onboard observation, robot action, transition, reward, discount and initial-state distribution. Although both phases involve RL for the policy, the adaptation modules in Phase II are trained by supervised L₁ embedding losses. Figure 2 preserves the complete computation and gradient-flow diagram.

Phase I: privileged encoders and policy
The shape encoder consumes recent ground-truth particle positions/position changes and corresponding robot actions. Its embedding feeds a dynamics encoder together with privileged object mass and position. The policy uses the resulting dynamics embedding and the current visual observation to produce full-DoF actions. The experiments also specify joint-angle observations.
The policy and both encoders are trained end-to-end by RL. This makes the embedding task-relevant but does not force its coordinates to equal physical material parameters. Keeping two encoders allows separate supervision of shape and dynamics during the next phase.
Phase II: adaptation modules
The shape adaptation module replaces particle inputs with recent depth images, joint angles and actions, regressing its output toward the privileged shape embedding with L₁ loss. The dynamics adaptation module similarly predicts the privileged dynamics embedding from observation/action history and the inferred shape representation.
Dynamics-module gradients are stopped before entering the shape module, and RL gradients are stopped before entering either adaptation module. The policy is then fine-tuned on the predicted latents. These restrictions help preserve separate supervisory targets; they do not prove that shape and dynamics factors are statistically disentangled.
Test-time deployment
The robot maintains a buffer of ten recent observation–action pairs. Shape and dynamics embeddings are updated every five timesteps to encourage short-term behavioral consistency, while the policy uses its current observation and latest embedding. Network weights stay fixed: “online adaptation” is recurrent inference of context, not online gradient learning.
Figure 6 shows selected latent coordinates across objects and an object whose effective bending behavior changes with rotation. The illustrated coordinate changes with the interaction, supporting a qualitative connection to softness. There is no calibrated stiffness target, correlation coefficient, controlled material-identification error or proof that this coordinate represents softness independently of shape, pose or action history. The camera runs at 3 Hz, but complete action/update latency is not reported.

IV Experiments
Robot and simulation
TIAGo has 22 controlled degrees of freedom: three for the mobile base, fourteen across two arms, one torso, two head and two grippers. A head-mounted RGB-D camera supplies 224×224 depth images at 3 Hz; the policy also receives joint angles. The authors report autonomous execution without human intervention or QR markers.
Training uses OmniGibson/BEHAVIOR-1K with a matching robot model, success reward of 1 and a small distance-based shaping reward. Real tables, objects and lighting are held out from simulation. Each task is evaluated 20 times with different objects and initial locations. Training-set size, network dimensions, optimizer/RL details, compute, action frequency and repeated-seed variability are not specified sufficiently for a complete reproduction.

Tasks and success definitions
Insertion requires picking an elongated object and placing either end into a container within 300 seconds. Covering requires picking a planar object and covering at least 90% of a container opening within the same limit. Real test objects span 20 categories per task; containers span 20 categories in the insertion description. These are 20 trials per task, not 20 trials for every category combination.
The complete Figure 4 inventory includes soft ropes/cloth as well as comparatively rigid items such as a marker, hammer, lid and cardboard. Thus the benchmark spans stiffness rather than consisting exclusively of highly deformable objects. The source says instance variation supplies diversity without explicit pose/stiffness/friction parameter randomization; it does not quantify coverage of those physical ranges. Some figure panels label the insertion task “1D Threading,” while the text and success rule call it insertion; the original labels are retained.

Baselines and ablations
DMfD uses simulated expert demonstrations; DDOD’s rope descriptors and fabric correspondences underpin the other comparison. The DDOD controller is scripted instead of using its original human-demonstration procedure. Both baselines use RAPiD for grasping/navigation before their manipulation stage, so the reported comparison is against adapted hybrid systems, not untouched original baselines.
No-Adapt removes both adaptation modules. No-Shape retains dynamics adaptation but removes shape adaptation. E2E keeps the broader architecture but skips privileged pretraining and L₁ imitation, training by end-to-end RL. These ablations test useful components/training choices together; they do not isolate a universal requirement for exactly this architecture.
Comparison with baselines
Table I reports insertion/covering success of 85/80% for RAPiD, 15/5% for DMfD and 10/25% for DDOD. Totals are 33/40 (82.5%), 4/40 (10%) and 7/40 (17.5%), respectively. Against the stronger baseline on each task, the absolute gains are 70 points for insertion and 55 for covering; the 65-point gain is the aggregate comparison with DDOD. The introduction’s “65%+” claim should not be interpreted as at least 65 percentage points on each task.
Qualitative failures involve segmentation or correspondence errors under gripper occlusion, deformation and oblique views, plus insufficient dynamic adaptation even when perception works. Figure 5 contrasts hanging a rope with inserting an adapter, and sweeping a towel with placing a hat. These examples are consistent with adaptive behavior but do not isolate perception, planning and dynamics inference as separate causes.
The table supplies exact counts but no uncertainty intervals, formal significance tests, per-category replication or training-seed variability. Large observed differences support this experimental setup; they do not establish generalization across all deformable manipulation or all unseen embodiments.

Removing adaptation
No-Adapt scores 7/20 insertion and 5/20 covering, totaling 30%. The full method’s 82.5% is 52.5 percentage points higher. The authors describe rigid-like handling of soft objects in the ablation. This supports the value of history-based adaptation within the tested system, while leaving architecture/capacity and training-budget controls unspecified.
Removing shape inference
No-Shape scores 7/20 insertion and 9/20 covering, totaling 40%, a 42.5-point deficit from RAPiD. It can grasp and approach the target but often fails at the final dynamic manipulation. The result supports adding shape-history supervision; it does not show that the inferred representation reconstructs the complete physical shape or material model.
End-to-end training
E2E scores 5/20 insertion and 4/20 covering, totaling 22.5%, a 60-point deficit. The authors attribute this to difficult RL optimization over high-dimensional histories and a failure to learn useful dynamics latents. The table establishes the performance gap, but no detailed convergence curves or gradient diagnostics verify that causal explanation. Privileged pretraining plus supervised adaptation is effective here; end-to-end RL is not ruled out with other architectures, budgets or optimization methods.
V Conclusion
RAPiD demonstrates simulation-trained latent adaptation on two real mobile-manipulation tasks without real robot demonstrations. Its strengths are the complete embodied experiment, explicit privileged-to-observed training path and substantial ablation gaps. Remaining limits include only 40 evaluation trials, altered baselines, unquantified training/physical coverage, sparse implementation detail and qualitative rather than calibrated evidence for material inference. Broader task, embodiment and real-data comparisons remain open.