Literature
Display
← All papers

VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon

Yi Pan · Miao Pan · Qi Lu · Jiaming Huang · Man Zhang · Siteng Huang · Xin Li · Jie Zhang · Yongliang Shen · Xuhong Zhang · Wenqi Zhang

· arXiv preprint ↗

Robotics8 min text · CondensedOriginal paper ↗
Contents

At a glance

VLA-Corrector monitors a robot while it executes a predicted action chunk. A separately trained visual-latent dynamics model detects persistent drift, discards pending actions, and guides the next policy query toward recovery. The task-fine-tuned VLA backbone stays frozen. Simulation and robot experiments report better success per policy call, although gradient guidance increases inference time.

1 Introduction

Action chunks amortize expensive policy inference but leave the robot executing old commands while the scene changes. Short fixed horizons restore feedback at the cost of more policy calls. VLA-Corrector instead adapts the executed horizon: it retains a chunk while visual evolution looks consistent, interrupts sustained deviations, and corrects the next replan. The external monitor needs demonstration-based training; freezing the backbone does not make the whole method training-free.

Refer to caption
Figure 1: Open-loop vs. Closed-loop execution. The top row (H=10H=10) illustrates how blindly executing long action chunks leads to compounding errors, causing the robot to get stuck during a drawer-opening task. In contrast, the bottom row (H=1H=1) demonstrates strict closed-loop execution, which maintains environmental reactivity and yields a smooth, successful manipulation.
Original figure
Figure 2: Performance–efficiency trade-off across fixed action horizons. Smaller horizon achieves higher success rates, while larger horizon preserves the chunking efficiency.

2 Preliminaries

The visual encoder supplies a latent observation to a language-conditioned policy. Each query generates C actions, of which the controller normally executes H before replanning. The corrective module operates during that execution window. It uses predicted versus observed changes in latent features as a consistency signal; this signal is not a ground-truth failure label or a safety certificate.

Equation 1
At=[at,at+1,…,at+C−1]∼πθ(⋅∣Ztreal,l),A_{t}=[a_{t},a_{t+1},\dots,a_{t+C-1}]\sim\pi_{\theta}(\cdot\mid Z_{t}^{\mathrm{real}},l), (1)

3 VLA-Corrector: A Lightweight Corrective Inference Framework

The framework combines an external dynamics predictor, an online deviation monitor, event-triggered truncation, and gradient guidance for recovery. The project repository supplies the implementation.

Refer to caption
Figure 3: Overview of VLA-Corrector. Starting from a standard chunked VLA pipeline (A), we add a Latent-space Vision Monitor (LVM) that detects persistent execution drift and triggers an interrupt event (B). The event truncates stale actions and switches the next replan from normal flow matching to OGG-guided flow matching (C). OGG uses the expected and observed latent evolution to guide the replan back toward a recoverable trajectory (D).

3.1 Training the External Latent Dynamics Corrector

After task fine-tuning, the VLA encoder and policy are frozen. A residual MLP learns the short-horizon change in visual features from the current latent and executed action. The main formulation combines residual-magnitude error with cosine-direction error. Appendix C describes a cosine-based loss for its deployed correctors; these reported training configurations should be distinguished.

Equations 2–4
Ztreal=ℰ⁡(ot),Zt+kreal=ℰ⁡(ot+k),Δ​Zt+k∗=Zt+kreal−Ztreal,Z_{t}^{\mathrm{real}}=\mathcal{E}(o_{t}),\qquad Z_{t+k}^{\mathrm{real}}=\mathcal{E}(o_{t+k}),\qquad\Delta Z_{t+k}^{*}=Z_{t+k}^{\mathrm{real}}-Z_{t}^{\mathrm{real}}, (2)
Δ​Z^t+k=Mϕ​(Ztreal,at).\Delta\hat{Z}_{t+k}=M_{\phi}(Z_{t}^{\mathrm{real}},a_{t}). (3)
ℒcorr=‖Δ​Z^t+k−Δ​Zt+k∗‖22+β⁡[1−CosSim⁡(Δ​Z^t+k,Δ​Zt+k∗)],\mathcal{L}_{\mathrm{corr}}=\left\|\Delta\hat{Z}_{t+k}-\Delta Z_{t+k}^{*}\right\|_{2}^{2}+\beta\left[1-\mathrm{CosSim}\left(\Delta\hat{Z}_{t+k},\Delta Z_{t+k}^{*}\right)\right], (4)

3.2 Latent Visual Dynamics for Online Anomaly Detection

The monitor compares the direction of the predicted latent residual with the direction actually observed k steps later. One minus their cosine similarity becomes the inconsistency score. The model learns local demonstrated dynamics rather than reconstructing future images or modeling the entire environment.

Equation 5
Et=1−CosSim⁡(Δ​Zt+kexp,Δ​Zt+kreal),CosSim⁡(𝐮,𝐯)=𝐮⊤​𝐯/(‖𝐮‖​‖𝐯‖).E_{t}=1-\mathrm{CosSim}\!\left(\Delta Z_{t+k}^{\mathrm{exp}},\Delta Z_{t+k}^{\mathrm{real}}\right),\qquad\mathrm{CosSim}(\mathbf{u},\mathbf{v})=\mathbf{u}^{\top}\mathbf{v}/(\|\mathbf{u}\|\,\|\mathbf{v}\|). (5)

3.3 Event-Triggered Truncation under Robust Online Monitoring

A rolling median and MAD set separate activation and deactivation thresholds. Sustained high scores trigger an interrupt, clear the remaining action queue, and mark the next query for correction. Hysteresis, a persistence counter, and cooldown suppress repeated interventions. Their delays and the prediction interval still limit response speed.

Equations 6–7
MAD=median⁡(|Ei−Me|),Ei∈𝐄W.\mathrm{MAD}=\mathrm{median}\!\left(|E_{i}-M_{e}|\right),\qquad E_{i}\in\mathbf{E}_{W}. (6)
Ton=Me+λon​MAD,Toff=Me+λoff​MAD,λon>λoff,T_{\mathrm{on}}=M_{e}+\lambda_{\mathrm{on}}\mathrm{MAD},\qquad T_{\mathrm{off}}=M_{e}+\lambda_{\mathrm{off}}\mathrm{MAD},\qquad\lambda_{\mathrm{on}}>\lambda_{\mathrm{off}}, (7)

3.4 Online Gradient Guidance for Corrective Inference

OGG differentiates through the corrector during flow-based action generation, using the flow-matching framework. [26] It estimates a clean action sequence from the current flow state, predicts the latent effect of its first action, and guides that effect toward an expected continuation corrected for the observed deviation. Only the query immediately following an interrupt receives this guidance; later queries return to ordinary inference unless another interrupt occurs.

Equations 8–11
Δ​Z^act=Mϕ​(Ztreal,a^t).\Delta\hat{Z}_{\mathrm{act}}=M_{\phi}(Z_{t}^{\mathrm{real}},\hat{a}_{t}). (8)
Δ​Zcorr=Δ​Zexp−Δ​Zdev.\Delta Z_{\mathrm{corr}}=\Delta Z_{\mathrm{exp}}-\Delta Z_{\mathrm{dev}}. (9)
ℒOGG=1−CosSim⁡(Δ​Z^act,Δ​Zcorr),\mathcal{L}_{\mathrm{OGG}}=1-\mathrm{CosSim}(\Delta\hat{Z}_{\mathrm{act}},\Delta Z_{\mathrm{corr}}), (10)
vτguide=vτ−η​∇vτℒOGG,Aτ−Δ​τ=Aτ−Δ​τ​vτguide.v_{\tau}^{\mathrm{guide}}=v_{\tau}-\eta\nabla_{v_{\tau}}\mathcal{L}_{\mathrm{OGG}},\qquad A^{\tau-\Delta\tau}=A^{\tau}-\Delta\tau v_{\tau}^{\mathrm{guide}}. (11)
Table 1: Cross-architecture generalization on MetaWorld. Success rate (%, ↑\uparrow) across difficulty splits. Green arrows show absolute improvements over the corresponding baseline.
Backbone Method Easy↑\uparrow Medium↑\uparrow Hard↑\uparrow Very Hard↑\uparrow Avg.↑\uparrow
π0.5\pi_{0.5} Baseline 70.5 45.0 38.3 41.0 48.70
+ VLA-Corrector 83.2 ↑ 12.7\uparrow\,12.7 61.7 ↑ 16.7\uparrow\,16.7 47.5 ↑ 9.2\uparrow\,9.2 65.0 ↑ 24.0\uparrow\,24.0 64.35 ↑ 15.65\uparrow\,15.65
SmolVLA Baseline 81.3 53.6 51.7 61.0 61.90
+ VLA-Corrector 83.4 ↑ 2.1\uparrow\,2.1 56.0 ↑ 2.4\uparrow\,2.4 64.2 ↑ 12.5\uparrow\,12.5 63.0 ↑ 2.0\uparrow\,2.0 66.65 ↑ 4.75\uparrow\,4.75
X-VLA Baseline 72.5 46.4 48.3 55.0 55.55
+ VLA-Corrector 74.4 ↑ 1.9\uparrow\,1.9 50.0 ↑ 3.6\uparrow\,3.6 50.0 ↑ 1.7\uparrow\,1.7 64.0 ↑ 9.0\uparrow\,9.0 59.60 ↑ 4.05\uparrow\,4.05
Table 2: Sample efficiency on LIBERO. Success rate (%, ↑\uparrow). Green arrows compare VLA-Corrector against the few-shot fine-tuned baseline.
Model Object↑\uparrow Spatial↑\uparrow Goal↑\uparrow Long↑\uparrow Avg.↑\uparrow
π0.5\pi_{0.5} (Full Fine-tuned) 99.4 98.2 97.8 92.4 96.95
π0.5\pi_{0.5} (Few-shot Fine-tuned) 97.8 95.4 96.2 86.6 94.00
π0.5\pi_{0.5} (Few-shot) + VLA-Corrector 99.8 ↑ 2.0\uparrow\,2.0 100.0 ↑ 4.6\uparrow\,4.6 98.0 ↑ 1.8\uparrow\,1.8 93.4 ↑ 6.8\uparrow\,6.8 97.80 ↑ 3.80\uparrow\,3.80

4 Experiments

Experiments use MetaWorld and LIBERO, compare π0.5, SmolVLA, and X-VLA, and test a physical PiPER arm. Success, policy-query counts, recovery, and inference time measure different aspects of performance. The π0.5 backbone and LIBERO benchmark are described in [15] and [27].

4.1 Main Results

Table 1 reports average MetaWorld gains of 15.65 percentage points for π0.5 (48.70% to 64.35%), 4.75 for SmolVLA, and 4.05 for X-VLA. The separate horizon sweep in Table 4 reports π0.5 at H=50 improving from 48.7% to 58.7%, with calls falling from 5.15 to 4.98. For SmolVLA at H=10, success rises from 61.9% to 73.0% while calls fall from 19.27 to 15.64. The tables report different evaluation settings and should not be combined into one result.

On LIBERO, the few-shot baseline rises from 94.0% to 97.8%, compared with 96.95% for the fully fine-tuned baseline. The corrector also trains on demonstrations. More corrector data helps, with diminishing returns around 60–80% of the post-holdout training pool.

Original figure
Original figure
Figure 4: Performance–efficiency analysis on π0.5\pi_{0.5}. Left: performance–efficiency trade-off across action horizons. Right: success-per-call efficiency. VLA-Corrector improves success rate across action horizons and yields consistent efficiency gains.
Table 3: Data efficiency of corrector training on MetaWorld. Success rate (%, ↑\uparrow). We compare the open-loop baseline at horizon 50 with VLA-Corrector trained using different fractions of demonstration trajectories. Green/red arrows indicate absolute change in average success over the baseline.
Method / Ratio Easy↑\uparrow Medium↑\uparrow Hard↑\uparrow Very Hard↑\uparrow Avg.↑\uparrow
Baseline 70.54 45.00 38.33 41.00 48.72
VLA-Corrector (r=0.2r=0.2) 71.07 49.55 36.67 36.00 48.32 ↓ 0.40\downarrow\,0.40
VLA-Corrector (r=0.4r=0.4) 70.36 45.91 38.33 42.00 49.15 ↑ 0.43\uparrow\,0.43
VLA-Corrector (r=0.6r=0.6) 70.89 52.73 39.17 46.00 52.20 ↑ 3.48\uparrow\,3.48
VLA-Corrector (r=0.8r=0.8) 71.07 53.64 38.33 46.00 52.26 ↑ 3.54\uparrow\,3.54
VLA-Corrector (r=1.0r=1.0) 73.21 55.91 44.17 44.00 54.32 ↑ 5.60\uparrow\,5.60
Table 4: Full performance–efficiency trade-off across VLA backbones and action horizons. Success rate, average policy calls per episode, and relative success-per-call efficiency gains are reported. VLA-Corrector consistently improves task success while maintaining or reducing policy calls in most settings.
Model Horizon Baseline Ours (LVM+OGG) Improvements
Succ (%)↑\uparrow Calls ↓\downarrow Succ (%)↑\uparrow Calls ↓\downarrow Δ\DeltaSucc (%)↑\uparrow Δ\DeltaCalls ↓\downarrow Eff. Gain (%)↑\uparrow
π0.5\pi_{0.5} 10 64.50 20.41 72.40 17.64 ↑ 7.90\uparrow\,7.90 ↓ 2.77\downarrow\,2.77 +29.9%
20 61.40 10.77 69.10 9.95 ↑ 7.70\uparrow\,7.70 ↓ 0.82\downarrow\,0.82 +21.8%
30 56.10 7.97 63.50 7.92 ↑ 7.40\uparrow\,7.40 ↓ 0.05\downarrow\,0.05 +13.9%
40 54.45 5.96 60.80 6.04 ↑ 6.35\uparrow\,6.35 ↑ 0.08\uparrow\,0.08 +10.2%
50 48.72 5.15 58.70 4.98 ↑ 9.98\uparrow\,9.98 ↓ 0.17\downarrow\,0.17 +24.6%
SmolVLA 10 61.90 19.27 73.00 15.64 ↑ 11.10\uparrow\,11.10 ↓ 3.63\downarrow\,3.63 +45.3%
20 58.90 11.32 68.90 9.50 ↑ 10.00\uparrow\,10.00 ↓ 1.82\downarrow\,1.82 +39.4%
30 58.10 7.90 65.20 7.60 ↑ 7.10\uparrow\,7.10 ↓ 0.30\downarrow\,0.30 +16.6%
40 56.80 5.64 67.20 5.13 ↑ 10.40\uparrow\,10.40 ↓ 0.51\downarrow\,0.51 +30.0%
50 54.20 4.86 62.90 4.68 ↑ 8.70\uparrow\,8.70 ↓ 0.18\downarrow\,0.18 +20.6%
X-VLA 4 68.50 46.58 72.00 35.20 ↑ 3.50\uparrow\,3.50 ↓ 11.38\downarrow\,11.38 +39.1%
8 67.80 23.18 72.10 19.48 ↑ 4.30\uparrow\,4.30 ↓ 3.70\downarrow\,3.70 +26.5%
16 55.50 13.71 60.90 13.92 ↑ 5.40\uparrow\,5.40 ↑ 0.21\uparrow\,0.21 +8.1%
24 52.90 10.13 59.20 9.92 ↑ 6.30\uparrow\,6.30 ↓ 0.21\downarrow\,0.21 +14.3%
32 44.00 8.61 54.40 8.34 ↑ 10.40\uparrow\,10.40 ↓ 0.27\downarrow\,0.27 +27.6%
  • •

    Δ\DeltaSucc: absolute success-rate improvement. Δ\DeltaCalls: absolute change in policy calls per episode. Eff. Gain: relative improvement in success-per-call over the baseline.

4.2 Mechanism Analysis

Failed episodes have a heavier high-inconsistency tail. Manual phase labels place 83.7% of truncations in precision-sensitive phases and 16.3% in tolerant phases; this association does not itself prove causality. After matched interruptions, OGG improves the reported recovery rate by an average 0.23. Recovery here means the score drops below the deactivation threshold within ten steps, rather than necessarily completing the task. A controlled grasping example illustrates eventual task recovery.

Original figure
Original figure
Figure 5: LVM detection analysis. Left: distribution of the inconsistency score EtE_{t}, where successful episodes concentrate at low values and failed episodes show a heavier high-score tail. Right: interrupt frequency, where failed episodes trigger more interrupt events than successful ones.
Refer to caption

Truncation Distribution
by Task Phase
Phase Rate Critical 83.7% Non-critical 16.3% 5.1×\times more truncations occur in critical phases than in non-critical phases.

Figure 6: Task-phase analysis of LVM-triggered truncation. We manually divide MetaWorld trajectories into critical and non-critical phases. Left: representative trajectories show that truncation is triggered much more frequently during critical phases. Right: 83.7% of truncations occur in critical phases, while only 16.3% occur in non-critical phases.
Original figure
Figure 7: Post-interrupt recovery. OGG-guided inference consistently outperforms standard inference.
Refer to caption
Figure 8: Controlled recovery case. Given the same initial state and detected grasping error, the monitored baseline continues the original chunk and fails (top), while VLA-Corrector truncates stale actions, replans with OGG, and completes the task (bottom).

4.3 Real-World Evaluation

Nine tasks on a six-degree-of-freedom PiPER arm cover pick-and-place, precise alignment, and disturbance recovery. Each method receives 20 trials per task with the same task-fine-tuned π0.5 backbone. Average success increases from 55.6% to 73.3%, with the largest gain in disturbance recovery. Table 5 gives binomial confidence intervals; the appendix reports individual tasks and a different across-task variability statistic.

Refer to caption
Figure 9: Real-world disturbance recovery demo. A human shifts the blue bowl during execution, requiring the robot to recover from an outdated action chunk.
Table 5: Real-world evaluation on AgileX PiPER. Success rate (%, ↑\uparrow) over three task groups. Each group contains three tasks with 20 trials per task. We report 95% binomial confidence intervals.
Method Pick-place↑\uparrow Alignment↑\uparrow Disturbance↑\uparrow Avg.↑\uparrow
π0.5\pi_{0.5} Baseline 70.0 ±\pm 11.6 56.7 ±\pm 12.5 40.0 ±\pm 12.4 55.6 ±\pm 7.3
+ VLA-Corrector 78.3 ±\pm 10.4 ↑ 8.3\uparrow\,8.3 73.3 ±\pm 11.2 ↑ 16.6\uparrow\,16.6 68.3 ±\pm 11.8 ↑ 28.3\uparrow\,28.3 73.3 ±\pm 6.5 ↑ 17.7\uparrow\,17.7

4.4 Ablation Studies

Truncation alone raises the component-ablation average from 48.70% to 60.35%; adding OGG yields 64.35%. An external detector also outperforms a jointly fine-tuned auxiliary detector, 64.35% versus 49.55%, under the same guidance setting. The proposed explanation—avoiding disruption to backbone representations—is a hypothesis. Moderate guidance and a roughly 40-million-parameter monitor work best in the reported sweeps.

Table 6: Component ablation on MetaWorld. Success rate (%, ↑\uparrow). Green arrows indicate average improvement over the baseline.
Variant Easy↑\uparrow Medium↑\uparrow Hard↑\uparrow Very Hard↑\uparrow Avg.↑\uparrow
Baseline (Open-loop) 70.5 45.0 38.3 41.0 48.70
+ Truncation Only 81.8 53.6 50.0 56.0 60.35 ↑ 11.65\uparrow\,11.65
+ Truncation + OGG 83.2 61.7 47.5 65.0 64.35 ↑ 15.65\uparrow\,15.65
Table 7: Coupled vs. decoupled detection on MetaWorld. Success rate (%, ↑\uparrow). Both variants use OGG; only the detector design differs.
Detector Easy Medium Hard Very Hard Avg.
Internal Head + OGG 65.2 46.0 36.5 50.5 49.55
Decoupled LVM + OGG 83.2 ↑ 18.0\uparrow\,18.0 61.7 ↑ 15.7\uparrow\,15.7 47.5 ↑ 11.0\uparrow\,11.0 65.0 ↑ 14.5\uparrow\,14.5 64.35 ↑ 14.80\uparrow\,14.80

5 Conclusion

Monitoring and targeted replanning improve the tested action-chunked policies while keeping their backbone weights fixed during correction. Benefits depend on suitable demonstrations, informative visual features, recoverable disturbances, and actions the backbone can represent. Higher success per query does not imply lower total computation.

Appendix A Related Work

The related work connects generative robot policies, action-horizon selection, and recovery from distribution shift.

A.1 Generative VLA Models

Vision-language-action models combine broad visual and language representations with robot action generation. Diffusion and flow models express complex action distributions but make frequent closed-loop queries costly, motivating a small external monitoring module.

A.2 Action Chunk and Horizon Trade-off

Chunking reduces policy-call frequency and supports temporal coherence, while a fixed long horizon delays reactions. Prior work changes chunking or query schedules; this paper combines an event-triggered horizon with guidance specifically for the following recovery query.

A.3 Failure Recovery in Visuomotor Policies

Recovery approaches expand training data through intervention, learn separate recovery policies, or use search and distribution constraints. VLA-Corrector trains a local latent predictor and guides the existing policy. It still relies on learned dynamics and the frozen policy’s available behaviors.

Appendix B Method Details

The appendix specifies the monitoring state machine and runtime parameters.

B.1 Details of Event-Triggered Truncation

Equation (12) increments the persistence counter above the activation threshold, resets it below the lower threshold, and holds it between thresholds. Reaching the patience value truncates the queue. The surrounding prose describes consecutive high-score steps, but the displayed rule permits intervening scores between thresholds; the original equation is retained to expose that distinction.

Equations 12–13
𝐄W={Et−w+1,…,Et}.\mathbf{E}_{W}=\{E_{t-w+1},\dots,E_{t}\}.
Me=median⁡(𝐄W),MAD=median⁡(|Ei−Me|),Ei∈𝐄W.M_{e}=\mathrm{median}(\mathbf{E}_{W}),\qquad\mathrm{MAD}=\mathrm{median}\!\left(|E_{i}-M_{e}|\right),\quad E_{i}\in\mathbf{E}_{W}.
Ton=Me+λon​MAD,Toff=Me+λoff​MAD,λon>λoff.T_{\mathrm{on}}=M_{e}+\lambda_{\mathrm{on}}\mathrm{MAD},\qquad T_{\mathrm{off}}=M_{e}+\lambda_{\mathrm{off}}\mathrm{MAD},\qquad\lambda_{\mathrm{on}}>\lambda_{\mathrm{off}}.
ct={ct−1+1,if ​Et>Ton,0,if ​Et<Toff,ct−1,otherwise.c_{t}=\begin{cases}c_{t-1}+1,&\text{if }E_{t}>T_{\mathrm{on}},\\ 0,&\text{if }E_{t}<T_{\mathrm{off}},\\ c_{t-1},&\text{otherwise}.\end{cases} (12)
ct≥p,c_{t}\geq p, (13)
Hadaptive=h<H.H_{\mathrm{adaptive}}=h<H.

B.2 LVM and OGG Runtime Parameters

The reported defaults are a 15-score window, MAD multipliers of 3 and 2, patience of five, five safe steps for reset, and a ten-step interrupt cooldown. Guidance strength is 1. OGG applies to a single post-interrupt replan, after which normal inference resumes.

Appendix C Training and Implementation Details

Training and evaluation details distinguish backbone fine-tuning, external corrector training, and online inference.

C.1 Benchmarks and Evaluation Protocol

MetaWorld results average tasks within four difficulty groups, generally with 20 episodes per task. The appendix specifies 10% validation and 10% test episodes before subsampling the remaining training pool, with three split seeds for the data-scaling analysis. The main text calls the held-out 20% validation; both descriptions leave an 80% training pool, but their holdout labels differ.

C.2 External Corrector Architecture

An action embedding is concatenated with the current visual latent and passed through four residual hidden layers of width 2048. Depending on feature and action dimensions, the predictor has approximately 38–42 million parameters. It predicts a feature residual rather than an absolute future feature vector.

C.3 Corrector Training

The corrector uses frozen demonstration features, AdamW, learning rate 0.0003, weight decay 0.0001, cosine scheduling to 1% of the initial rate, and up to 30 epochs with early-stopping patience five. Ratio sweeps use training batches of 128 and evaluation batches of 256; deployed h1-k10 correctors use batches of 512 and a cosine-based loss. The retained equation defines optimization-step counts.

Equation
steps​per​epoch=⌈NtrainB⌉,total​steps=30×steps​per​epoch,\mathrm{steps\ per\ epoch}=\left\lceil\frac{N_{\mathrm{train}}}{B}\right\rceil,\qquad\mathrm{total\ steps}=30\times\mathrm{steps\ per\ epoch},

C.4 Compute Resources

Simulation jobs use a server with eight A100 GPUs, each with 40 GB memory. Monitoring adds an external forward pass; recovery queries additionally backpropagate guidance. Being smaller than a VLA backbone does not eliminate that inference overhead.

Appendix D Additional Experimental Results

Additional results study guidance strength, monitor capacity, domain transfer, and wall-clock cost.

D.1 Ablation Sensitivity

Guidance strength 1 gives the best average in the sweep; stronger guidance harms some difficult tasks. Increasing monitor size from 10 to 40 million parameters improves performance, while 160 million brings little further average gain.

Table 8: Ablation on OGG guidance strength η\eta. Success rate (%, ↑\uparrow) on MetaWorld. The highlighted row is the default setting.
Guidance Strength Easy↑\uparrow Medium↑\uparrow Hard↑\uparrow Very Hard↑\uparrow Avg.↑\uparrow
η=0.1\eta=0.1 82.7 56.8 45.2 66.0 62.68
η=1\eta=1 (default) 83.2 ↑ 0.5\uparrow\,0.5 61.7 ↑ 4.9\uparrow\,4.9 47.5 ↑ 2.3\uparrow\,2.3 65.0 ↓ 1.0\downarrow\,1.0 64.35 ↑ 1.67\uparrow\,1.67
η=10\eta=10 82.5 56.8 38.3 65.0 60.65 ↓ 3.70\downarrow\,3.70
η=100\eta=100 80.9 55.0 36.7 63.0 58.90 ↓ 5.45\downarrow\,5.45
Table 9: Ablation on LVM capacity. Success rate (%, ↑\uparrow) on MetaWorld. The highlighted row is the default setting.
Monitor Capacity Easy↑\uparrow Medium↑\uparrow Hard↑\uparrow Very Hard↑\uparrow Avg.↑\uparrow
LVM-10M 78.5 52.4 39.8 55.6 56.58
LVM-40M (default) 83.2 ↑ 4.7\uparrow\,4.7 61.7 ↑ 9.3\uparrow\,9.3 47.5 ↑ 7.7\uparrow\,7.7 65.0 ↑ 9.4\uparrow\,9.4 64.35 ↑ 7.77\uparrow\,7.77
LVM-160M 82.8 ↓ 0.4\downarrow\,0.4 62.1 ↑ 0.4\uparrow\,0.4 48.0 ↑ 0.5\uparrow\,0.5 64.2 ↓ 0.8\downarrow\,0.8 64.28 ↓ 0.07\downarrow\,0.07

D.2 Cross-Domain Corrector Generalization

A LIBERO-trained corrector improves the MetaWorld baseline from 48.7% to 51.8%, while a MetaWorld-trained corrector reaches 58.7% in this comparison. This shows limited cross-domain transfer and a substantial benefit from domain-matched demonstrations.

Table 10: Cross-domain corrector generalization on MetaWorld. Success rate (%, ↑\uparrow) of the same π0.5\pi_{0.5} MetaWorld baseline equipped with correctors trained from different demonstration domains. Green arrows indicate absolute improvement over the baseline.
Method Easy↑\uparrow Medium↑\uparrow Hard↑\uparrow Very Hard↑\uparrow Avg.↑\uparrow
π0.5\pi_{0.5} Baseline 70.5 45.0 38.3 41.0 48.7
+ LIBERO-trained Corrector 70.9 ↑ 0.4\uparrow\,0.4 50.5 ↑ 5.5\uparrow\,5.5 40.7 ↑ 2.4\uparrow\,2.4 45.0 ↑ 4.0\uparrow\,4.0 51.8 ↑ 3.1\uparrow\,3.1
+ MetaWorld-trained Corrector 74.1 ↑ 3.6\uparrow\,3.6 53.8 ↑ 8.8\uparrow\,8.8 52.9 ↑ 14.6\uparrow\,14.6 54.0 ↑ 13.0\uparrow\,13.0 58.7 ↑ 10.0\uparrow\,10.0

D.3 Inference-Time Overhead

Enabling OGG increases total measured inference time to 1.62–1.68 times that of the same pipeline without guidance. The reported mean cost per executed step rises from 12.32 to 20.25 milliseconds; a guided query takes 588.52 milliseconds versus 278.01 for a standard query. Guidance is intermittent, but the measured total overhead remains material. Policy-call efficiency and wall-clock efficiency must therefore be assessed separately.

Table 11: Overall inference-time summary by backbone on MetaWorld. Timing is accumulated over all evaluated episodes and difficulty splits. “w/o OGG” disables guidance while keeping the same action-chunk inference pipeline. The ratio reports the wall-clock overhead of enabling OGG; lower is better (↓\downarrow).
Backbone Episodes Steps / Ep. Calls / Ep. w/o OGG s / Ep.↓\downarrow w/ OGG s / Ep.↓\downarrow Overhead↓\downarrow
π0.5\pi_{0.5} 1000 163.77 6.22 2.49 4.03 1.62×\times
SmolVLA 1000 158.75 5.67 2.13 3.51 1.65×\times
X-VLA 1000 177.92 10.30 1.54 2.59 1.68×\times
Overall 3000 166.81 7.39 2.06 3.38 1.64×\times
Table 12: Average per-step inference-time overhead. We report the average inference time per executed environment step. Although OGG-guided recovery queries are slower than standard inference, they are only invoked after interrupt events, so the amortized per-step overhead remains small.
Backbone w/o OGG ms / Step↓\downarrow w/ OGG ms / Step↓\downarrow Extra ms / Step↓\downarrow Relative Overhead↓\downarrow
π0.5\pi_{0.5} 15.23 24.62 +9.39 1.62×\times
SmolVLA 13.44 22.14 +8.70 1.65×\times
X-VLA 8.66 14.55 +5.89 1.68×\times
Overall 12.32 20.25 +7.93 1.64×\times
Table 13: Average inference time per MetaWorld task. Each task is evaluated with 20 episodes. The table reports the average total inference time per task and the average number of OGG recovery events per task.
Backbone Tasks Episodes / Task w/o OGG s / Task↓\downarrow w/ OGG s / Task↓\downarrow OGG Events / Task
π0.5\pi_{0.5} 50 20 49.87 80.63 71.28
SmolVLA 50 20 42.66 70.29 62.24
X-VLA 50 20 30.81 51.76 119.76

Appendix E Real-World Details

The physical experiments document task definitions, trial conditions, observed failures, and example recovery sequences.

E.1 Robot Platform and Task Suite

Both methods use the same camera observations, instructions, action horizon, and task-fine-tuned backbone. Success requires completing the task without a human reset after execution starts. The nine tasks progress from ordinary placements to precise alignment and once-per-trial disturbances.

Table 14: Real-world task suite on AgileX PiPER. The benchmark contains three task groups and nine tasks in total. The disturbance group reuses manipulation skills from the first two groups but introduces online pose changes during precision-sensitive phases.
Group Task Description
Pick-and-place Cube-to-region Pick up a cube and place it inside a marked target region.
Cube-to-drawer-top Pick up a cube and place it on top of a small drawer platform.
Object-to-container Pick up a cube and place it into a specific bowl.
Alignment Corner placement Place a cube precisely at the upper-left corner of the drawer top.
Square-hole insertion Align and insert a small cube into a square hole or slot.
Edge alignment Align an object with a narrow marked edge or target strip.
Disturbance recovery Moving-object grasp Move the target object when the gripper is about to grasp it.
Moving-placement target Move the placement region after the object is grasped.
Moving-insertion target Move the square hole or alignment target before insertion.

E.2 Per-Task Real-World Results

Per-task results show modest gains for ordinary pick-and-place and larger gains for alignment and shifted targets. Table 15 summarizes group and overall variation as standard deviation across tasks; these values differ in meaning from Table 5’s binomial confidence intervals.

Table 15: Per-task real-world success rates. Success rate (%, ↑\uparrow) over 20 trials per task on the AgileX PiPER 6-DoF arm. Group and overall rows report mean ±\pm standard deviation across tasks. Green arrows indicate absolute improvement over the π0.5\pi_{0.5} baseline.
Group Task π0.5\pi_{0.5} Baseline↑\uparrow + VLA-Corrector↑\uparrow
Pick-and-place Cube-to-region 75.0 85.0 ↑ 10.0\uparrow\,10.0
Cube-to-drawer-top 70.0 80.0 ↑ 10.0\uparrow\,10.0
Object-to-container 65.0 70.0 ↑ 5.0\uparrow\,5.0
Group Avg. 70.0 ±\pm 5.0 78.3 ±\pm 7.6 ↑ 8.3\uparrow\,8.3
Alignment Corner placement 60.0 75.0 ↑ 15.0\uparrow\,15.0
Square-hole insertion 50.0 70.0 ↑ 20.0\uparrow\,20.0
Edge alignment 60.0 75.0 ↑ 15.0\uparrow\,15.0
Group Avg. 56.7 ±\pm 5.8 73.3 ±\pm 2.9 ↑ 16.6\uparrow\,16.6
Disturbance recovery Moving-object grasp 45.0 75.0 ↑ 30.0\uparrow\,30.0
Moving-placement target 40.0 70.0 ↑ 30.0\uparrow\,30.0
Moving-insertion target 35.0 60.0 ↑ 25.0\uparrow\,25.0
Group Avg. 40.0 ±\pm 5.0 68.3 ±\pm 7.6 ↑ 28.3\uparrow\,28.3
Overall Avg. across 9 tasks 55.6 ±\pm 13.8 73.3 ±\pm 7.1 ↑ 17.7\uparrow\,17.7

E.3 Real-World Protocol and Task Details

Each task supplies 10–30 teleoperated demonstrations for backbone fine-tuning. Initial poses vary within a small feasible region. Placement tasks use a marked area, drawer platform, or bowl; alignment tasks use a corner, square opening, or narrow edge. Disturbances move an object before grasping or a target before placement/insertion, once per trial, within view and a recoverable range. Both methods use the same phase-based disturbance rules.

E.4 Observed Real-World Failure Modes

Failures remain when a target becomes unreachable, a grasp or timing leaves too little room to recover, contact geometry or friction defeats visual correction, or occlusion and poor contrast obscure the scene. Without force feedback, visual consistency cannot resolve all contact failures. Guidance cannot create recovery behaviors absent from the backbone and supplies no formal collision-avoidance guarantee.

E.5 Real-World Demonstration Examples

The retained sequences show recovery after a cube, a bowl, or a drawer target is moved during execution. These examples illustrate the mechanism; the trial tables provide the quantitative evidence.

Refer to caption
Figure 10: Demo: Moving-object grasp. The robot picks up the cube and places it into the white bowl, while a human manually changes the cube’s position during the process.
Refer to caption
Figure 11: Demo: Moving-placement target. The robot picks up the cube and places it into the blue bowl, while a human manually changes the bowl’s position during the process.
Refer to caption
Figure 12: Demo: Moving-insertion target. The robot picks up the cube and places it at the upper-left corner of the drawer top, while a human manually changes the drawer’s position during the process.
Concept demonstration

Open image ↗