Literature
Display
← All papers

ReDex: Repairing Sim-to-Real Dexterous Policies by Finger-Level Compliant Interaction

Jinzhou Li · Hadi Tabatabaee · Kelin Yu · Yuyin Sun · Cheng-Hao Kuo · Roberto Martín-Martín · Nima Fazeli · X. Alice Wu · Xianyi Cheng

Preprint posted · arXiv (preprint) ↗

Robotics5 min text · CondensedOriginal paper ↗
Contents

At a glance

ReDex adapts a simulation-trained dexterous policy by letting a person physically correct selected fingers while the frozen policy controls the others. Recorded fingertip forces help reconstruct training targets, and behavior cloning produces a standalone force-conditioned policy. Across two real flipping objects, success rises from 14% to 86%. Across three screwdrivers, average rotation progress rises from 26.0% to 95.3%; that latter metric includes partial rotations and is not a full-task success rate.

I Introduction

Simulation can learn useful finger coordination while still transferring poorly at local contact transitions. ReDex assumes those local failures can be corrected without replacing the whole behavior. Its three stages are base-policy training, selective compliant physical correction, and distillation of corrected rollouts with force-aware targets. The human retains control over which fingers are corrected, and the final policy learns from successful object-specific executions.

Refer to caption
Fig. 1: ReDex overview. ReDex repairs local sim-to-real failures through finger-level physical correction and reconstruction of training targets. We show (a) Screwdriver Rotation and (b) cellphone Object Flipping. Top: real-world contact failures during base policy execution. Bottom: physical correction of selected fingers. RL denotes the base policy. The corrected rollouts are relabeled with force-informed targets and distilled into a standalone force-conditioned policy via behavior cloning (BC).
Refer to caption
Fig. 2: ReDex pipeline. Top: ReDex follows a three-stage pipeline: (1) train a proprioception-only dexterous policy in simulation, (2) collect real-world finger-level physical corrections through a compliant framework while the base policy controls the remaining fingers, and (3) train a standalone force-conditioned policy from the reconstructed real-world rollouts. Bottom: We detail the correction and supervision-reconstruction process: (2a) finger-level shared control enables selective physical guidance without interrupting the remaining hand motion; (2b) measured fingertip forces are mapped to force-informed joint targets that encode the desired contact load; (2c) these targets are smoothly blended with the evolving base policy targets at intervention onset and release; and (2d) the reconstructed targets form the training supervision for the final policy, which uses proprioception and fingertip force for autonomous deployment.

II Related Work

Related work covers simulation-trained dexterity, correction-based policy adaptation, and tactile supervision.

II-A Learning Dexterous Manipulation via RL

Simulation provides parallel reinforcement learning, privileged state, and domain randomization, but friction, contact, and actuation errors still impair transfer. ReDex keeps the learned coordination during data collection and repairs local failures on hardware instead of retraining the simulator policy.

II-B Learning from Human Correction

Interactive imitation and residual policies learn from human intervention. Existing dexterous interfaces may coordinate the full hand or combine human wrist control with learned finger motions. ReDex instead reduces stiffness on selected fingers and lets the operator guide only those contacts while the remaining fingers continue under the base policy.

II-C Learning with Contact Feedback

Contact information can enter through demonstrations, physical guidance, tactile simulation, or post-training adaptation. ReDex uses real measured forces in two distinct ways: offline target reconstruction and online observations for the distilled policy. It requires no tactile simulator but does require suitable hardware sensing and corrective demonstrations.

III Method

The method separates the controller used to collect corrections from the standalone learned controller deployed afterward.

III-A Sim-to-Real Policy Training

A privileged PPO teacher is trained in Isaac Sim, followed by an adaptation module that estimates its latent from joint-position and target history. Deployment of this base policy uses proprioception only. Task-specific rewards and domain randomization cover dynamics, controller gains, and noise. One base policy per skill is reused across its real object variants without further simulation training.

III-B Finger-Level Compliant Correction

Keyboard commands select fingers for correction. Their proportional gains and torque limits are reduced, derivative gains use a guidance profile, and integral gains are zero. Crucially, selected position targets track measured joints instead of continuing to pull toward the base target. Unselected fingers retain nominal control, while the frozen base policy observes the whole corrected hand and keeps updating its commands. Release restores nominal control and the current base target. Logs retain joint states, base targets, forces, and intervention masks.

Equations 1–2
(kq,t)j={(kqcmp)j,(mt)j=1,(kqnom)j,(mt)j=0.(k_{q,t})_{j}=\begin{cases}(k_{q}^{\mathrm{cmp}})_{j},&(m_{t})_{j}=1,\\ (k_{q}^{\mathrm{nom}})_{j},&(m_{t})_{j}=0.\end{cases} (1)
at=mt⊙ath+(𝟏−mt)⊙atRL,a_{t}=m_{t}\odot a_{t}^{h}+(\mathbf{1}-m_{t})\odot a_{t}^{\mathrm{RL}}, (2)

III-C Reconstructing Force-informed Action Targets

After sensor taring and thresholding, corrected fingertip forces are transformed into the hand frame and mapped through Jacobian transposes to joint torques. A fixed diagonal virtual stiffness converts those torques into position offsets added to the measured correction pose. The scale is chosen as 0.05 through preliminary replay and then held fixed. These quasi-static force-informed targets supervise only intervened joints; other joints retain the base policy’s recorded targets. Minimum-jerk interpolation smooths transitions at correction boundaries. The reconstruction encodes contact load approximately; it is not a dynamic inverse model or a direct force command.

Equations 3–5
𝝉tf=∑i∈ℐt𝐉i,t⊤​𝐟i,t.\boldsymbol{\tau}^{f}_{t}=\sum_{i\in\mathcal{I}_{t}}\mathbf{J}_{i,t}^{\top}\mathbf{f}_{i,t}. (3)
Δ​atf\displaystyle\Delta a_{t}^{f} =𝐊f−1​𝝉tf,\displaystyle=\mathbf{K}_{f}^{-1}\boldsymbol{\tau}^{f}_{t}, (4)
atf\displaystyle a_{t}^{f} =qt+Δ​atf.\displaystyle=q_{t}+\Delta a_{t}^{f}.
Δ​𝐱i,tf=𝐉i,t​Δ​ai,tf=𝐉i,t​𝐊f−1​𝐉i,t⊤⏟𝐂x,i,t​𝐟i,t,\Delta\mathbf{x}_{i,t}^{f}=\mathbf{J}_{i,t}\Delta a_{i,t}^{f}=\underbrace{\mathbf{J}_{i,t}\mathbf{K}_{f}^{-1}\mathbf{J}_{i,t}^{\top}}_{\mathbf{C}_{x,i,t}}\mathbf{f}_{i,t}, (5)
Refer to caption
Fig. 3: Qualitative rollout. Each column shows four frames of autonomous execution after learning from human corrections, ordered from top to bottom. From left to right, we show Object Flipping with a cuboid and a cellphone, and Screwdriver Rotation with normal (N), small (S), and large (L) screwdrivers. Insets visualize changes in screwdriver orientation. T denotes the elapsed time, in seconds, from the start of the corresponding rollout.

III-D Learning Force-Conditioned Policy

Successful relabeled rollouts train a behavior-cloning policy from five-step histories of joint positions and fingertip forces. It predicts 16 joint-target steps, executes the first 12 at 20 Hz, then replans. The final policy no longer invokes a residual gate or the base policy. It can therefore reproduce corrections directly, but may also lose base-policy generalization outside its limited real training distribution.

Equation 6
ℒBC​(θ)=𝔼(xt,At⋆)∼𝒟​[‖πθ​(xt)−At⋆‖F2],\mathcal{L}_{\mathrm{BC}}(\theta)=\mathbb{E}_{(x_{t},A_{t}^{\star})\sim\mathcal{D}}\left[\left\|\pi_{\theta}(x_{t})-A_{t}^{\star}\right\|_{F}^{2}\right], (6)

IV Experiments

Experiments assess autonomous performance, the roles of tactile observations and target construction, tactile representations, and recorded human guidance effort.

IV-A Experimental Setup

All tests use a five-finger, 12-joint XHAND1 with 120 triaxial taxels per fingertip. Flipping succeeds only after a stable 90° reorientation; screwdriver progress is discretized in 90° increments toward 360°. Main results use 25 trials per object and method; ablations use separate ten-trial evaluations. Each object receives its own final policy, trained from 30 successful corrected episodes for flipping or ten for rotation. Residual baselines share rollouts, targets, and force observations but predict single-step corrections, while ReDex predicts chunks; they are reference comparisons rather than isolated architectural ablations.

TABLE I: Real-robot performance over 25 autonomous trials per object–method pair. Prog. is mean discretized rotation progress, quantized in 90∘90^{\circ} increments and reported as a percentage; SR is the number of successful trials. Screwdriver Overall is the mean across the three object variants, while Flipping Overall pools trials across the two object variants. Overall results are reported only when all corresponding object variants were evaluated.
Screwdriver Rotation (progress. %)    Object Flipping (successes / trials)   
Method Normal Small Large Overall Cuboid iPhone Overall
RMA [2] 26 33 19 26.0 4/25 3/25 7/50
Gated Corr. [7] 42 — — — 3/25 — —
Continuous Corr. [8] 36 — — — 1/25 — —
ReDex (Ours) 95 99 92 95.3 20/25 23/25 43/50
TABLE II: Component ablation on real-robot Object Flipping (Cuboid). All variants use motion supervision. Checkmarks denote enabled components; SR gives successes over 10 matched trials.
Variant Force obs. Virtual target Transition blending SR.
Motion only – – – 4/10
w Tactile obs. ✓\checkmark – – 6/10
w/o virtual target ✓\checkmark – ✓\checkmark 7/10
w/o blending ✓\checkmark ✓\checkmark – 6/10
Ours ✓\boldsymbol{\checkmark} ✓\boldsymbol{\checkmark} ✓\boldsymbol{\checkmark} 9/10

V Results

On cuboid flipping, ReDex reaches 20/25 successes versus 4/25 for the base policy; cellphone flipping reaches 23/25 versus 3/25. Combined success is 43/50, or 86%, versus 7/50, or 14%. Normal, small, and large screwdriver progress reaches 95%, 99%, and 92%, respectively. These are separate object-specific learned policies sharing a base skill, not zero-shot transfer of one final policy across objects. Gated and continuous residual baselines reach 42% and 36% normal-screwdriver progress and 3/25 and 1/25 cuboid successes. Their differing action structure and tuning prevent attributing all differences to a single component.

On the separate ten-trial cuboid ablation, motion-only supervision yields 4/10 successes, adding tactile observations gives 6/10, and the complete method gives 9/10. Removing force-informed targets gives 7/10; removing transition blending gives 6/10. Net-force features outperform raw taxels in this small study: net-force MLP/GRU encoders achieve 9/10 and 8/10, versus 6/10 and 2/10 with taxels. These results support the chosen compact representation but are too small to establish a universal tactile-encoding rule.

For 30 replay-validated cuboid trajectories, recorded guidance falls from 13.2 minutes for full kinesthetic teaching to 2.4 minutes for ReDex, an approximately 82% reduction. ReDex guides the thumb during 24% of retained rollout time; the base policy handles the rest. This accounting excludes failed attempts, resets, and replay validation, so it is not total data-collection labor. Screwdriver guidance occupies 72.0%, 38.8%, and 83.6% of retained rollout time for normal, large, and small variants. A comparable quantitative full-teaching result is unavailable for screwdriver rotation because demonstrations replayed unreliably.

Corrections must preserve useful coordination: excessive or mistimed finger motion can move the hand outside the base policy’s useful state distribution. Limited real-data distillation may narrow generalization. Magnetic sensor drift can corrupt both force labels and policy observations after initial taring, and the study covers only two skills on one hand platform. These constraints qualify claims of low-effort general adaptation.

TABLE III: Ablation of tactile observation, representation, and encoder on real-robot Object Flipping (Cuboid). All variants follow a matched 10-trial protocol. Net force stacks one fingertip triaxial resultant per fingertip.
Tactile Obs. Encoder Flip SR.
Taxels MLP 6/10
Taxels GRU 2/10
Net force GRU 8/10
Net force MLP 9/10
Refer to caption
Fig. 4: Human correction across objects. Bars show the share of recorded rollout time with at least one finger under human correction. Values inside the bars give human / total rollout time in minutes, rounded to 0.1 min. The five circles represent the thumb, index, middle, ring, and little fingers from left to right; blue marks guided fingers and gray marks fingers controlled by the base policy.
TABLE IV: Human guidance effort on Object Flipping (cuboid). Both methods yield 30 trajectories that successfully replay without human intervention. Reported rollout durations sum the recorded execution time of these trajectories; they exclude failed attempts, resets, and replay validation. Both methods use force-informed joint position targets for replay.
Full kinesthetic teaching Ours
Guided fingers Thumb, index Thumb
Replay-validated trajectories 30 30
Recorded rollout duration (min) 13.2 9.9
Human time (min) 13.2 2.4
Mean duration per trajectory (s) 26.4 19.8

VI Conclusion

Selective compliant correction and force-aware distillation improve two contact-rich skills without more simulation training. Extending the method to other sensors, more coupled hands, and whole-body tasks remains future work, as does preserving the base skill’s broader adaptability.

Acknowledgments

The acknowledgments credit discussions on primitive-skill training and feedback, along with proofreading and fabrication support.

Concept demonstration

Open image ↗