Literature
Display
← All papers

Never Too Late for Force: Accelerating VLA Post-Training with Reactive Force Injection

Yi Wang · Wendi Chen · Zimo Wen · Han Xue · Xueqi Li · Wenye Yu · Zhijie Chen · Hao Yang · Jun Lv · Chuan Wen · Cewu Lu

Preprint posted · CoRL 2026 (accepted) ↗

Robotics11 min text · CondensedOriginal paper ↗
Contents

Overview

LIFT adds force-aware reactive execution and online corrective post-training to a pretrained vision-language-action policy.

Refer to caption

Figure 1: LIFT overview. (Left) LIFT copies the pretrained base action expert to initialize a reactive action expert, with zero-initialized force-injected cross attention preserving the original policy’s output at initialization. (Center-left) Causal force memory encodes recent force history and injects it through cross attention for reactive within-chunk action updates. (Center-right) To address force distribution shift (force OOD), online DAgger collects on-policy contact corrections and updates the policy using mixed offline and online data. (Right) LIFT achieves higher peak and final performance than vision-only DAgger across the three evaluated manipulation tasks.

At a glance

LIFT adds a copied reactive action expert and causal force memory to π0.5, with a zero-initialized force path that preserves matched-input outputs at initialization. Offline visual demonstrations and online human corrections then jointly adapt the model. Three physical manipulation tasks show gains over vision-only online DAgger, especially for insertion and placement. Reactivity is not uniformly best: single-frame force performs better on towel folding. Generalization tests change appearance under fixed task geometry, and the implementation uses 10 Hz policy inputs and control.

1 Introduction

Force can reveal contact, seating, or jamming that is ambiguous in a wrist-camera image. Adding it after large-scale visual pretraining avoids requiring force data for the entire foundation-model corpus, but introduces new timing and distribution-shift problems. LIFT couples a causal action pathway with online corrective data so the model learns from contact states induced by its own behavior. Initialization equivalence protects the starting policy; preservation after learning must be judged empirically.

2 Related Work

The work combines reactive visual-force policies, general VLA models, and interactive post-training.

Force-augmented policies.

Prior force-reactive policies separate slow visual reasoning from fast contact-conditioned control. ImplicitRDP [11] uses force memory and causal attention; LIFT adapts related mechanisms to a pretrained mixture-of-transformers VLA with an additional action expert.

Vision-language-action models for manipulation.

Large policies such as π0.5 [5] primarily learn from visual and robot-motion data. Other recent systems add force through routing, adapters, prompts, or hybrid control. LIFT emphasizes a reactive branch that starts with the pretrained action behavior before force-enabled adaptation.

Post-training for robotic policies.

DAgger [23] and human-gated variants collect expert actions at learner-visited states. Compliant residual DAgger [30] applies this idea to contact-rich correction. LIFT uses online intervention data to adapt its full augmented VLA rather than only a small residual controller.

3 Problem Formulation

Stage one aligns the visual policy to tasks using handheld demonstrations without force. Stage two adds synchronized six-dimensional wrench measurements and human corrective commands from the robot. Training retains the visual dataset while expanding the correction dataset. The design goals are reactive force use, initial compatibility with the pretrained policy, and joint learning from heterogeneous data.

4 Methods

The architecture adds causal reactive decoding and force memory while reusing visual-language computation. Parameter copying and zero initialization preserve the initial interface; data masking and balanced sampling support subsequent adaptation.

4.1 Closing the loop with reactive force injection (O1)

The standard action expert predicts a whole chunk with full attention. A second branch supports causal updates within that chunk using recent force, while a cached visual-language prefix reduces repeated computation.

Refer to caption
Figure 2: LIFT architecture. LIFT grafts a reactive action expert beside pretrained π0.5\pi_{0.5} while keeping the original vision-language stack unchanged. Recent force is encoded as force memory and injected into the reactive branch through zero-initialized cross attention, enabling within-chunk action updates while preserving the pretrained prior at initialization.

O1.1 Reactive action expert.

The reactive expert shares the original action-token interface but uses a causal reactive stream. It remains connected to the original base stream through the shifted attention pattern, so it is not an independent small controller.

O1.2 Causal force-injected cross attention.

A causal encoder summarizes recent six-axis force/torque measurements. Cross-attention injects this memory only into reactive action tokens, with a mask offset chosen to align available force observations with execution after inference delay.

O1.3 Cached Vision-Language Slow Context.

The runtime computes and caches the vision-language prefix once per context update, then reevaluates the action pathways with the newest force history. This avoids rerunning the entire visual-language tower for every contact response, although both action experts are still computed.

4.2 Preserving the pretrained VLA prior at initialization (O2)

The added pathway is designed to reproduce the base policy at initialization for matched inputs. This architectural property does not guarantee that later full-network post-training preserves every original capability.

Original figure
Figure 3: Shifted causal attention with initialization equivalence. Each reactive token attends to the vision-language prefix, later base-action tokens, and the causal reactive prefix. Combined with the copied pretrained action expert, LIFT preserves initialization equivalence while enabling causal reactive updates.

O2.1 Make reactive action expert output-equivalent.

Copied weights initialize the reactive expert. At position i, its token attends to the visual-language prefix, later base-action tokens, and its own causal reactive prefix, while blocking the corresponding earlier base tokens and future reactive tokens. At initialization, the copied reactive prefix supplies the missing base context, making the effective context equivalent to full base attention under aligned representations.

O2.2 Zero-initialize cross attention output.

The force-attention output projection starts at zero, so its residual initially contributes nothing. Force influences actions only after training learns a nonzero projection. Together with copying and shifted attention, this prevents the newly attached sensor path from immediately perturbing the pretrained output.

4.3 Training with heterogeneous visual and force data (O3)

Both visual-only and force-enabled examples train one model. Explicit masks prevent missing force from being treated as a measured physical signal.

O3.1 Additive flow-matching objective.

An additive flow-matching objective trains both base and reactive action streams using separate noise samples at the same diffusion time. Gradients are accumulated jointly. The base stream remains necessary as context during inference, but only the reactive action is executed. Joint training anchors both branches to demonstrations; it does not freeze the base network.

Equation 1
ℒ(θ)=∥vθ(xτ,τ,o)−u∥22+∥vθr(xτr,τ,o,mt:t+H)−ur∥22.\mathcal{L}(\theta)\;=\;\big\lVert v_{\theta}(x_{\tau},\tau,o)-u\big\rVert_{2}^{2}\;+\;\big\lVert v_{\theta}^{\text{r}}(x_{\tau}^{\text{r}},\tau,o,m_{t:t+H})-u^{\text{r}}\big\rVert_{2}^{2}. (1)

O3.2 Selective force masking.

Visual-only samples contain shape-compatible zero placeholders, but encoded force and its gradients are masked. Corrective examples activate measured force and train the force pathway. This permits mixed data without inventing force labels for handheld demonstrations.

O3.3 Equal sampling of heterogeneous data.

Updates sample offline visual and online corrective data equally. This fixed 1:1 mixture retains task-alignment supervision while adding contact corrections; the appendix tests alternative ratios on towel folding.

4.4 Pipeline overview and system implementation

The system first trains with force masked, then alternates robot collection and cloud post-training. New completed correction episodes enter the online buffer and checkpoints periodically return to the rollout client. The base visual dataset remains available throughout the second stage.

Refer to caption
Figure 4: LIFT training pipeline. The two-stage system first aligns LIFT on vision-only demonstrations and then closes an online post-training loop between the robot and cloud trainer.

5 Experimental Results

Experiments compare learning curves, contact-memory ablations, appearance shifts, online versus fixed-buffer adaptation, and a custom lightweight residual alternative.

5.1 Experimental setup

A Flexiv Rizon 4S with a wrist camera and six-axis force sensor performs towel folding, book insertion, and Hanoi ring placement. Checkpoints receive three groups of ten autonomous trials, with 95% confidence intervals. Sample counts mean human-intervention frames, not total environment frames or demonstrations. Towel and book scores measure completed stages; Hanoi is binary success. Comparisons include vision-only DAgger, single-frame force, a fixed correction buffer, visual-only offline training, and a custom residual policy.

5.2 Does force accelerate or improve post-training? (Q1)

For towel folding, full LIFT peaks at 0.825 score with about 2,800 corrective frames, compared with 0.725 for vision-only online DAgger at about 3,100. The single-frame force variant performs better still, peaking at 0.95. Force therefore helps this task, while temporal reactivity is not necessary for the best reported result.

For book insertion, LIFT peaks at 0.6 stage score versus 0.4 for vision-only DAgger. For Hanoi, it reaches and retains 0.6 binary success, versus a vision-only peak of 0.3 and final 0.2. Contact feedback plausibly helps detect completed insertion and jamming, but the qualitative failure explanations are not separate causal ablations of individual contact events.

Original figure
Figure 5: Performance curves on three tasks as sample count increases. By leveraging reactive control and force memory, LIFT reaches higher peak performance with fewer samples.

5.3 Does reactivity matter beyond single-frame force? (Q2)

Reactive memory improves the harder contact decisions. Book insertion reaches 0.6 with full LIFT versus 0.4 for single-frame force, which can keep pushing after seating. On Hanoi, the single-frame variant peaks at 0.5 but later falls to 0–0.1, while full LIFT remains at 0.6. Force history and within-chunk updates change together in this comparison, so their individual contributions are not isolated. Towel folding favors the simpler force variant.

5.4 Does LIFT preserve original VLA generalization? (Q3)

The final model is tested with changed object appearance, tablecloth, or lighting and shows limited degradation in most conditions. Robot start poses and target locations are held fixed. These tests support appearance robustness within the three tasks, not unrestricted preservation of the original VLA’s general abilities or novel-layout transfer.

Original figure
Figure 6: Generalization of the final LIFT checkpoint. LIFT shows no substantial degradation under shifts of objects, tablecloths, and lighting.

5.5 Does online data matter for reactive force injection? (Q4)

Training the reactive model on a fixed correction buffer underperforms repeated online collection, reaching zero on book insertion. This supports the usefulness of corrections at newly visited contact states in this protocol. It does not show that all offline datasets are insufficient regardless of coverage or size.

5.6 Why use LIFT rather than a lightweight residual policy? (Q5)

A roughly 28-million-parameter force-reactive transformer added to a frozen base policy scores substantially below LIFT. The authors attribute this to a weak base requiring large corrections and sparse residual targets making timing difficult. The baseline is a custom design inspired by prior work, not a reproduction of those full systems. LIFT’s stronger result comes with full-network training and two action branches.

Original figure
Figure 7: Lightweight residual baseline on three tasks. The lightweight residual module substantially underperforms LIFT.

6 Conclusion and Limitations

The demonstrated benefit is adding contact feedback and online correction to a pretrained VLA for three physical tasks. Human monitoring and skilled takeover remain substantial burdens; intervention timing affects data quality. Evaluation uses one single-arm platform and sensor setup. Broader embodiments, sensor transfer, and reduced operator dependence remain future work.

Acknowledgments

The authors thank their collaborators and reviewers and acknowledge support from the listed Chinese education, provincial, laboratory, and XPLORER PRIZE programs.

Appendix A Latency-aligned causal mask details

A single-layer causal GRU and linear projection encode wrench history. Reactive queries attend only to force entries allowed by a latency-offset causal mask. The offset accounts for inference delay so the output targets the execution time rather than an outdated action. The configured offset is three steps; the paper does not provide a separate measured end-to-end latency study.

Equations 2–3
mt:t+H=Lineardm(GRUdh(Ft:t+H))∈ℝH×dm,\displaystyle m_{t:t+H}\;=\;\mathrm{Linear}_{d_{m}}\!\big(\mathrm{GRU}_{d_{h}}(F_{t:t+H})\big)\;\in\;\mathbb{R}^{H\times d_{m}}, (2)
Cross(qi,m0:H−1)\displaystyle\mathrm{Cross}(q_{i},m_{0:H-1}) =∑j=0H−1αi​j​mj​WV,\displaystyle=\sum_{j=0}^{H-1}\alpha_{ij}\,m_{j}W_{V}, (3)
αi​j\displaystyle\alpha_{ij} =exp⁡(si​j+bi​j(L))∑k=0H−1exp⁡(si​k+bi​k(L)),si​j=(qi​WQ)​(mj​WK)⊤dm,\displaystyle=\frac{\exp(s_{ij}+b^{(L)}_{ij})}{\sum_{k=0}^{H-1}\exp(s_{ik}+b^{(L)}_{ik})},\qquad s_{ij}=\frac{(q_{i}W_{Q})(m_{j}W_{K})^{\top}}{\sqrt{d_{m}}},
bi​j(L)\displaystyle b^{(L)}_{ij} ={0,j≤i−L,−∞,j>i−L.\displaystyle=\begin{cases}0,&j\leq i-L,\\ -\infty,&j>i-L.\end{cases}

Appendix B Pseudocode: cached reactive inference

Algorithm 1 retains the complete cached inference loop: compute the slow prefix, encode admissible recent force, reevaluate action experts, and execute the relevant reactive action while updating within a chunk.

Algorithm 1 Cached reactive inference with latency-aligned force injection.
1: policy πθ\pi_{\theta}, language instruction ℓ\ell, image stream, force stream, execution horizon HH, diffusion noise level NN, latency offset LL
2: t←0t\leftarrow 0
3: while task not done do
4:   It←GetImage⁡()I_{t}\leftarrow\mathrm{GetImage}()
5:   KVL,VVL←VLAprefix​(ℓ,It)K_{\mathrm{VL}},V_{\mathrm{VL}}\leftarrow\mathrm{VLA}_{\mathrm{prefix}}(\ell,I_{t}) ⊳\triangleright cache slow vision-language context
6:   xN,xNr∼𝒩⁡(0,𝐈)x_{N},x_{N}^{\mathrm{r}}\sim\mathcal{N}(0,\mathbf{I}) ⊳\triangleright initialize base and reactive action noise
7:   cache the latency-aligned causal mask b(L)b^{(L)} with bi​j(L)=−∞b^{(L)}_{ij}=-\infty for j>i−Lj>i-L
8:   ℱ←[]\mathcal{F}\leftarrow[\,] ⊳\triangleright clear force memory history for this chunk
9:   for i←0i\leftarrow 0 to H−1H-1 do
10:    ft+i←GetForceFrame⁡()f_{t+i}\leftarrow\mathrm{GetForceFrame}()
11:    append ft+if_{t+i} to ℱ\mathcal{F}
12:    mt:t+i←ForceEncoder(ℱ)m_{t:t+i}\leftarrow\mathrm{ForceEncoder}(\mathcal{F}) using (2)
13:    a^t+i←DenoiseAction(πθ,KVL,VVL,xN,xNr,mt:t+i,b(L))\hat{a}_{t+i}\leftarrow\mathrm{DenoiseAction}(\pi_{\theta},K_{\mathrm{VL}},V_{\mathrm{VL}},x_{N},x_{N}^{\mathrm{r}},m_{t:t+i},b^{(L)})
14:    Execute⁡(a^t+i)\mathrm{Execute}(\hat{a}_{t+i})
15:    if task done then
16:      break
17:    end if
18:   end for
19:   t←t+Ht\leftarrow t+H ⊳\triangleright refresh slow context, noise, and force history
20: end while

Appendix C Online DAgger protocol

The online protocol combines a fixed visual dataset with a growing intervention-only force dataset and periodically synchronizes model checkpoints.

Data sources.

The offline set is collected with a handheld gripper and camera. The online set contains synchronized force-enabled human corrections appended after completed rollouts.

Fixed data mixing.

Each training batch samples offline and online sources equally. The main comparisons keep this ratio fixed rather than adapting it automatically.

Synchronization.

The trainer checks for completed episodes every optimizer step and sends an updated checkpoint to the inference server every 100 steps. Collection and training therefore proceed asynchronously.

Human intervention.

An operator monitors rollouts and resets stalled, over-time, or excessive-force episodes. Hardware safety thresholds and human oversight remain external to the learned policy.

Appendix D Hardware and sensor setup

All physical tests use the same single-arm robot, wrist camera, force sensor, and bilateral teleoperation arrangement.

Robot.

A seven-degree-of-freedom Flexiv arm with a Robotiq 2F-85 gripper receives position targets at 10 Hz, tracked by its internal impedance controller. The learned policy is not demonstrated as a kilohertz force controller.

Sensing.

The built-in wrench sensor samples above 1,000 Hz but is downsampled and synchronized to 10 Hz for the policy. A wrist-mounted iPhone also supplies RGB at 10 Hz. Sensor acquisition rate and learned control rate are therefore different.

Teleoperation.

Flexiv TDK uses a second master arm for human control. Follower-arm contact forces are reflected to the master, allowing the operator to feel interactions while providing corrective motions.

Appendix E Dataset details

The two data sources share relative action supervision but differ in sensing and collection platform.

Stage-one offline data (stage-one handheld).

Handheld demonstrations record 10 Hz RGB and ARKit pose estimates in the iPhone camera frame, without force. Short static segments at the beginning and end are removed.

Stage-two online data (stage-two online correction data).

Robot collection sets the tool-center frame to the camera frame and records synchronized wrench, RGB, and robot commands. Only human-intervention portions enter the online training set; autonomous rollout frames do not count as corrective supervision.

Action representation.

Actions are relative and padded to the shared output-head dimension. The architecture table lists a 32-dimensional padded action vector.

Appendix F Training hyperparameters

The original tables specify a ten-action horizon, six-dimensional wrench, GRU hidden width 512, force-memory width 1,024, and latency offset three. Optimization uses batch size 32, up to 30,000 steps, a 10,000-step warmup, and AdamW. The full schedules and optimizer constants are retained for reproduction.

Table 1: Architecture hyperparameters.
Parameter Value
Backbone π0.5\pi_{0.5} with PaliGemma vision-language tower and action expert
Action chunk horizon 1010
Padded action dimension 3232
Force input dimension 66D wrench
Force encoder Single-layer causal GRU followed by a linear projection
Force encoder hidden width 512512
Force encoder output width dm=1024d_{m}=1024
Latency-alignment offset 33
Reactive expert initialization Copied from the base action expert
Cross-attention initialization Zero-initialized output projection
Table 2: Optimization hyperparameters.
Parameter Value
Batch size 3232
Total training steps 30,00030{,}000
Learning-rate schedule 10,00010{,}000-step warmup, cosine schedule
Peak and final learning rate ×10−55\!\times\!10^{-5}; decay horizon 1,000,0001{,}000{,}000
Optimizer AdamW
AdamW β1\beta_{1} 0.90.9
AdamW β2\beta_{2} 0.950.95
AdamW ε\varepsilon 10−810^{-8}
Weight decay 10−1010^{-10}
Gradient clipping norm 1.01.0

Appendix G Compute and reproducibility

The compute and implementation details qualify the practical cost of full VLA post-training.

Compute.

Each training run uses one node with eight NVIDIA A800 GPUs. This is considerably more compute than the lightweight residual baseline is intended to require.

Code release.

The implementation builds on openpi; the paper links its LIFT code repository. The edition preserves the original repository link.

Appendix H Residual baseline implementation

The custom residual baseline freezes the offline π0.5 and trains a causal transformer with force GRU to predict additive position corrections. It caches vision at 1 Hz and produces corrections at 10 Hz. Targets are expert-minus-base actions during interventions and zero otherwise, trained with mean-squared error and the same fixed data mixture. Its design and supervision differ from full-network LIFT adaptation.

Appendix I Task protocol and failure case analysis

Rollouts last at most two minutes. Towel trials start 5 cm above a designated corner; Hanoi begins with the ring already held above a 10 mm-diameter pole. These controlled initial conditions are part of the evaluation protocol.

Refer to caption
Figure 8: Task protocol and failure case analysis. Qualitative examples of towel folding, book insertion, and Hanoi ring placement.

Towel folding.

Towel scores are 0.25 for the first grasp, 0.5 for the first fold, 0.75 for the second grasp, and 1 for the second fold. Empty grasps, choosing the wrong second corner, and incorrect folds are distinct failure modes.

Book insertion.

Book scores are 0.5 after lifting and collision-free insertion, and 1 after the specified final spine push and disengagement. Side collisions, early release, over-pushing, incomplete insertion, and failure to disengage explain partial rather than complete outcomes.

Hanoi ring placement.

Hanoi receives one only when the ring is fully seated at the pole base. Angled insertion, jamming, oscillation, and drift are failures. Its score is directly a success rate, unlike the other tasks’ stage scores.

Appendix J Training-step view of learning curves

Optimizer-step curves complement corrective-frame curves. Each online experiment takes roughly two to three hours and collects 20–30 episodes, with only intervention frames used for online training. These timings do not imply equal total compute across architectures.

Original figure
Figure 9: Learning curves on three tasks as optimizer training steps increase. The horizontal axis counts optimizer updates; training steps are positively correlated with training time.

Appendix K Additional ablations

An additional towel-folding study tests the balance between retained offline demonstrations and online corrections.

K.1 How does the offline:online data ratio affect post-training? (Q6)

Online-only training performs worse than 1:1 and 1:2 offline-to-online mixtures, and 1:1 improves faster in this test. This supports retaining visual task data during force adaptation; it does not establish one universal optimal ratio.

Original figure
Figure 10: Offline data ratio ablation. Mixed training outperforms online-only training; the main protocol uses fixed 1:11{:}1 sampling.

Appendix L Generalization condition details

The generalization conditions change one listed appearance factor at a time while retaining the original start pose and target location.

Refer to caption
Figure 11: Generalization task settings. The three panels show the towel folding, book insertion, and Hanoi ring placement setups used for shifted-condition evaluation.

Towel folding.

Towel tests replace the towel with a blue one of the same style, change to a smoother yellow patterned tablecloth, or add purple side lighting. Geometry remains fixed.

Book insertion.

Book tests substitute a gray book, use the alternate tablecloth, or add the same purple light. These are object-appearance and scene-appearance changes within the insertion task.

Hanoi ring placement.

Hanoi tests change tablecloth or side lighting while leaving initial pose and target location unchanged. The result should be interpreted as controlled appearance robustness rather than broad spatial generalization.

Concept demonstration

Open image ↗