Literature
Display
← All papers

VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids

Zichao Hu · Zifan Xu · Dongsik Chang · He Yin · Linh Tran · Roberto Martín-Martín · Peter Stone · Jingyu Qiao · Joydeep Biswas

· IROS 2026 ↗

Robotics9 min text · CondensedOriginal paper ↗
Contents

Overview

This edition covers the complete nine-page arXiv v4, dated 21 July 2026. All seven figures and five tables are retained, including the opening figure hidden inside the HTML title-note markup and three appendix tables omitted from the HTML section hierarchy. Zichao Hu is the correspondence author.

Complete original figure with every panel, label and legend.
Fig. 1: We present VOFA, a visual goal-conditioned humanoid loco-manipulation system capable of pushing objects with unknown physical properties. The system adapts to different goal positions (a,b), object masses (c), and center-of-mass configurations (d) while maintaining stable, closed-loop control.

At a glance

VOFA combines a depth-based high-level pushing policy with a frozen force-adaptive whole-body controller on the Booster T1 humanoid. Privileged-state reinforcement learning trains a teacher; DAgger distills it into a student using onboard proprioception, low-resolution depth histories and a supplied goal position.

The student reaches 91.38% reported overall simulation success. Real goal-position tests yield 25 successes in thirty trials; a separate 17-kg box condition succeeds three times in five attempts. Relative goals come from motion capture during deployment, while object feedback is visual.

I Introduction

Pushing a large object to a location requires locomotion, contact stability and feedback under unknown mass/friction. The hierarchy separates choosing robot–object approach geometry from maintaining whole-body stability under interaction forces.

The high-level policy learns to move around the box before contact, using an alignment reward rather than manually staged approach/push training. A force-perturbation-trained controller handles contact, while visual augmentation supports transfer from simulation.

Selected examples show recovery after lateral box displacement, off-center loading and goal changes. The study concerns one box-shaped object and supplied spatial goals; broader rearrangement with multiple objects remains future work.

Complete original figure with every panel, label and legend.
Fig. 2: VOFA Design. VOFA adopts a hierarchical architecture that combines a high-level visuomotor policy with a force-adaptive whole-body controller [12] for humanoid visual object–goal pushing. The high-level policy is trained in a teacher–student framework: a teacher policy is trained with privileged observations and goal positions using PPO [15], and a vision-based student policy is distilled via DAgger [13] using onboard sensory inputs and goal information. Training uses four reward terms—reach, push, track, and alignment—to promote robust closed-loop object–goal interaction.

II Related Work

Related work separates reusable humanoid whole-body controllers from autonomous perception-driven locomotion and manipulation. VOFA combines these lines for visually guided pushing under varied object properties.

II-A Humanoid Whole Body Control

Motion-imitation controllers provide coordinated locomotion and balance, while teleoperation systems extend them to object interaction. FALCON specifically trains against unknown end-effector forces. VOFA uses this controller as a frozen low-level component and learns a goal-directed visual command policy above it.

II-B Sim-to-Real Reinforcement Learning for Autonomous Humanoid Loco-Manipulation

Earlier dynamic object-goal pushing and other legged manipulation systems often use privileged object state. VisualMimic, VIRAL and pixel-to-action systems introduce visual control. This paper focuses on the combination of goal variation, onboard object sensing and mass/friction variation; it does not rerun all those systems as direct benchmarks.

III Method

At each control step, map proprioceptive/depth histories and a desired object-goal position to a high-level command, then map that command through the whole-body controller to joint targets. Both policy loops run at 50 Hz, but new depth observations are processed at 5 Hz.

The deployed student receives onboard object observations, while the goal is an external relative-position input. The exact policy/history notation remains below; the simplified low-level conditional notation does not enumerate all controller internals.

Equations
πH(𝒖t∣𝒐t−h1:tprop,𝒐t−h2:tdepth,𝒑tgoal)\pi_{H}(\boldsymbol{u}_{t}\mid\boldsymbol{o}_{t-h_{1}:t}^{\text{prop}},\boldsymbol{o}_{t-h_{2}:t}^{\text{depth}},\boldsymbol{p}_{t}^{\text{goal}})
πL​(𝒂t∣𝒖t)\pi_{L}(\boldsymbol{a}_{t}\mid\boldsymbol{u}_{t})

III-A High-Level Policy Design and Training

Train a teacher with privileged state, then imitate its command choices with a visual student. This separates discovering useful pushing behavior from learning a representation robust to deployment sensing.

Teacher Policy Design

Use PPO with an asymmetric actor–critic. Actor inputs include base linear/angular velocity, projected gravity, previous high-level command, upper-body joint state, both end-effector–object relative positions, object orientation and goal position, expressed in the robot base frame.

The critic additionally receives contact, object center of mass, mass, dimensions and linear/angular velocity. Commands stay within the low-level controller’s valid range. Table III retains the authors’ exact observation dimensions, including unusual entries that are not silently corrected.

Equations
𝒐tpriv=(𝒐tprop-priv,𝒐texte-priv,𝒑tgoal)\boldsymbol{o}_{t}^{\text{priv}}=(\boldsymbol{o}_{t}^{\text{prop-priv}},\boldsymbol{o}_{t}^{\text{exte-priv}},\boldsymbol{p}_{t}^{\text{goal}})
𝒐tprop-priv=(𝒗t,𝝎t,𝒈t,𝒖t−1teacher,𝒒tupper,𝒒˙tupper)\boldsymbol{o}_{t}^{\text{prop-priv}}=(\boldsymbol{v}_{t},\boldsymbol{\omega}_{t},\boldsymbol{g}_{t},\boldsymbol{u}^{\text{teacher}}_{t-1},\boldsymbol{q}_{t}^{\text{upper}},\dot{\boldsymbol{q}}_{t}^{\text{upper}})
𝒐texte-priv=(𝒑tee,r-obj,𝒑tee,l-obj,𝑹t)\boldsymbol{o}_{t}^{\text{exte-priv}}=(\boldsymbol{p}_{t}^{\text{ee,r-obj}},\boldsymbol{p}_{t}^{\text{ee,l-obj}},\boldsymbol{R}_{t})
𝒐tpriv-critic=(ct,𝒑tobj,mtobj,𝒅tobj,𝒗tobj,𝝎tobj),\boldsymbol{o}_{t}^{\text{priv-critic}}=\big(c_{t},\;\boldsymbol{p}_{t}^{\text{obj}},\;m_{t}^{\text{obj}},\;\boldsymbol{d}_{t}^{\text{obj}},\;\boldsymbol{v}_{t}^{\text{obj}},\;\boldsymbol{\omega}_{t}^{\text{obj}}\big),

Student Policy Design

DAgger trains the student on its visited states with teacher commands. Temporal 1D convolutions encode proprioceptive history; a CNN encodes each depth frame; an MLP fuses those features for command prediction.

The student omits privileged object state and base linear velocity, using angular velocity, gravity, previous command and joint positions/velocities. Maintain three recent depth frames, process depth at 5 Hz and reduce 640×360 input to 32×32. Table IV specifies a 67-dimensional proprioceptive input and 3×32×32 image history.

Printed history indices t−50:t and t−3:t are retained exactly; the prose explicitly specifies three buffered depth frames, so inclusive endpoint notation should not be used to invent a fourth frame.

Equations
(𝒐t−50:tprop,𝒐t−3:tdepth)(\boldsymbol{o}_{t-50:t}^{\text{prop}},\boldsymbol{o}_{t-3:t}^{\text{depth}})
𝒐tprop=(𝝎t,𝒈t,𝒖t−1student,𝒒t,𝒒˙t)\boldsymbol{o}_{t}^{\text{prop}}=(\boldsymbol{\omega}_{t},\boldsymbol{g}_{t},\boldsymbol{u}_{t-1}^{\text{student}},\boldsymbol{q}_{t},\dot{\boldsymbol{q}}_{t})

III-B Low-Level Policy Design and Training

Pretrain FALCON for 10,000 PPO epochs with AMASS motions and a curriculum of increasing external forces at the end effectors, then freeze it during high-level training. Force adaptation is learned through perturbations, without a reported explicit object-mass estimator or new force-sensor input.

The high-level command controls locomotion and residual upper-body joint targets. Fix walking mode, root height and waist yaw to defaults 1, 0.62 and 0. The complete command definitions are preserved below; Table III gives thirteen previous-command dimensions.

Equations
𝒖t=(𝒖tlower,𝒒tupper)\boldsymbol{u}_{t}=(\boldsymbol{u}_{t}^{\text{lower}},\boldsymbol{q}_{t}^{\text{upper}})
𝒖tlower=(𝒗tlin,ang,ϕtstance,htroot,ωtyaw)\boldsymbol{u}_{t}^{\text{lower}}=(\boldsymbol{v}_{t}^{\mathrm{lin,ang}},\phi_{t}^{\mathrm{stance}},h_{t}^{\mathrm{root}},\omega_{t}^{\mathrm{yaw}})

III-C Reward Design

Equations 1–2 reward both hands reaching the box via the harmonic mean of two exponential distance scores, discouraging a single hand from dominating. Equation 3 rewards proximity of object and goal. Equation 4 rewards head alignment only while the object lies inside the specified field-of-view limit.

Equation 5 favors the robot being opposite the goal across the object, encouraging repositioning before pushing. Its normalized-angle convention is not fully specified. Table V retains all shaping scales, weights and penalties for command changes, object tilt and upper-joint actions.

Equations 1–5
rteei=exp⁡(−∥𝒑tee,i−obj∥2σreach2),i∈{l,r}.r^{\text{ee}_{i}}_{t}=\exp\!\left(\frac{-\lVert\boldsymbol{p}_{t}^{\text{ee,i}-\text{obj}}\rVert^{2}}{\sigma^{2}_{\text{reach}}}\right),\quad i\in\{l,r\}. (1)
Exact original PDF equation 2, including every term and delimiter.
Selectable original MathML
Rreach=HarmonicMean⁡(rteel,rteer).R_{\text{reach}}=\mathrm{HarmonicMean}\!\left(r^{\text{ee}_{l}}_{t},\,r^{\text{ee}_{r}}_{t}\right). (2)
Rpush=exp⁡(−‖𝒑tgoal−𝒑tobj‖2σpush2).R_{\text{push}}=\exp\!\left(\frac{-\|\boldsymbol{p}_{t}^{\text{goal}}-\boldsymbol{p}_{t}^{\text{obj}}\|^{2}}{\sigma_{\text{push}}^{2}}\right). (3)
Rtrack={exp⁡(−θt2σtrack2),if ​|θt|≤θfov,0,otherwise,R_{\text{track}}=\begin{cases}\exp\!\left(\frac{-\theta_{t}^{2}}{\sigma^{2}_{\text{track}}}\right),&\text{if }|\theta_{t}|\leq\theta_{\text{fov}},\\ 0,&\text{otherwise},\end{cases} (4)
Ralign=exp⁡(−(1−θtalign)2σalign2),R_{\text{align}}=\exp\!\left(\frac{-(1-\theta_{t}^{\text{align}})^{2}}{\sigma^{2}_{\text{align}}}\right), (5)
TABLE I: Performance under different initial goal positions.
Overall Front Lateral Rear
Method SR(%) ↑\uparrow SR(%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C. ↓\downarrow SR (%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C.(%) ↓\downarrow SR(%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C.(%) ↓\downarrow
Teacher w/ FA (ours) 97.84 98.12 0.72 0.96 97.58 1.00 1.16 97.34 1.12 1.08
Teacher w/o FA 76.9476.94 83.7883.78 2.922.92 5.525.52 79.0079.00 4.884.88 6.586.58 71.8671.86 6.466.46 7.387.38
Student w/ FA (ours) 91.38 91.97 1.84 2.42 91.40 1.30 3.22 90.22 1.92 3.98
Student w/o FA 59.3259.32 71.1671.16 2.682.68 7.007.00 64.9064.90 4.524.52 7.787.78 56.0656.06 5.645.64 8.848.84
TABLE II: Performance under different box mass.
Light Medium Heavy Extra Heavy (Unseen)
Method SR(%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C.(%) ↓\downarrow SR(%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C.(%) ↓\downarrow SR (%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C.(%) ↓\downarrow SR(%) ↑\uparrow Pre-C.(%) ↓\downarrow Post-C.(%) ↓\downarrow
Teacher w/ FA (ours) 97.40 1.12 1.34 98.22 0.92 0.74 97.46 1.06 1.08 87.92 0.88 4.16
Teacher w/o FA 82.1882.18 5.405.40 9.009.00 80.9880.98 5.085.08 6.066.06 69.7669.76 4.644.64 6.066.06 53.6853.68 4.784.78 5.665.66
Student w/ FA (ours) 93.50 1.46 3.06 93.30 1.40 2.74 89.16 1.22 3.22 69.00 1.30 6.30
Student w/o FA 70.9870.98 6.606.60 9.469.46 62.6862.68 5.985.98 7.647.64 48.6648.66 6.566.56 9.589.58 32.0032.00 5.365.36 9.949.94

III-D Environment Setup and Domain Randomization

Train in Isaac Gym with box mass 1–8 kg and friction 0.1–1.0. Sample goals over the full 360° at radius 2.5–3.5 m; start the box 1.8–2.2 m away within ±60° of the robot’s view.

Match ZED 2i intrinsics, then perturb camera translation by ±3 cm on each axis and orientation by ±5° yaw/pitch and ±2° roll. Add upper-image far-depth patches, spatially correlated Gaussian depth noise and pixel dropout to mimic background/sensor artifacts.

Figure 3’s labels identify (a) unaugmented simulation, (b) augmented simulation and (c) reality. Some prose cross-references reverse (b)/(c); the original panels and captions are retained with their printed labels.

Complete original figure with every panel, label and legend.
Fig. 3: Egocentric vision of the humanoid robot. During training, we apply three vision augmentations: far-plane depth perturbation, correlated depth noise, and pixel dropout.
Selectable original panel labels

(a) Sim Vanilla

(b) Sim w/ Aug.

(c) Real

Complete original figure with every panel, label and legend.
Fig. 4: Force-Adaptive Controller Ablation. With the force-adaptive controller, the robot stably pushes the object using its end-effector (a). Without it, the robot struggles to apply consistent forces, often resorting to kicking, which results in unstable behavior and a higher risk of falling.
Selectable original panel labels

(a) w/ Force-Adaptive Controller (ours)

(b) w/o Force-Adaptive Controller

IV Experimental Results

Simulation evaluates goal angle, object mass and force-adaptation ablations. Real tests evaluate goal configurations, mass changes and visual randomization. Separate qualitative demonstrations inspect response to disturbances, asymmetric loading and sequential goals.

Complete original figure with every panel, label and legend.
Fig. 5: Object-Goal Alignment Reward Ablation. The alignment reward enables the policy to reposition around the object before pushing it toward the goal (a). Without it, the robot tends to make premature contact and fails to push the object to the goal (b).
Selectable original panel labels

(a) w/ Align Reward (ours)

(b) w/o Align Reward

IV-A Performance Across Initial Configurations and Masses

Success requires placing the box within 0.3 m of the goal and maintaining that position for two seconds. Pre-C. and Post-C. count robot falls before and after object contact. Remaining failures include failure to reach the goal; fall percentages are not the complete failure rate.

Across front/lateral/rear angle groups, the force-adaptive teacher exceeds 97% and the student exceeds 90% success. Reported overall rates are 97.84%/91.38%. Some overall entries do not equal the unweighted means of their three displayed groups; simulation sample counts and aggregation details are not supplied.

For light 1–3, medium 3–5 and heavy 5–8 kg boxes, student success is 93.50%, 93.30% and 89.16%. Unseen 8–12 kg boxes yield 69.00% versus 32.00% without force adaptation. The heavy condition therefore falls below 90%; the broad headline does not apply to every mass group.

Complete original figure with every panel, label and legend.
Fig. 6: Real World Deployment Results. VOFA demonstrates strong sim-to-real transfer performance. Real-world experiments highlight the benefits of visual randomization across different goal positions (a) and the force-adaptive low-level controller across varying object masses (b).
Selectable original panel labels

(a) Different Goal Positions

(b) Different Object Masses

Complete original figure with every panel, label and legend.
Fig. 7: Closed-Loop Demonstration. The policy reacts to external disturbances (a) and supports sequential goal switching without reset (b), illustrating closed-loop control.
Selectable original panel labels

(a) External perturbation recovery.

(b) Sequential multi-goal pushing.

IV-B Effect of Force-Adaptive Whole-Body Control on VOFA

Remove random end-effector force perturbations during low-level pretraining, then train corresponding high-level teacher/student policies. This keeps the hierarchical task design while changing the controller’s force experience.

Without force adaptation, reported overall teacher/student success falls to 76.94%/59.32%, with poorer lateral/rear and heavier-box performance. Figure 4 illustrates unstable kicking rather than steady hand pushing. The comparison establishes the value of the trained controller within this system; it is not a direct measured-force feedback ablation.

IV-C Impact of Object–Goal Alignment Reward

Without alignment shaping, the reach reward encourages premature contact from the wrong side, blocking useful exploration of walking around the box. Figure 5 shows the resulting failure and the aligned policy’s repositioning behavior.

This ablation is qualitative in the paper: no separate numeric success table, repeated-run distribution or reward-weight sensitivity curve is supplied.

IV-D Sim-to-Real Transfer Performance

Deploy on Booster T1 with a 5-kg cardboard box approximately 1.0×1.0×0.7 m. Motion capture supplies the relative goal; the policy still locates/interacts with the object from depth. A separate localization system is suggested as a possible replacement, but is not evaluated.

Ten trials per goal condition give 9/10 front, 8/10 lateral and 8/10 rear successes. In a separate fixed 4-m front-goal experiment, add 0, 2.5, 4.5 or 12 kg payload to the 5-kg box, giving total masses 5, 7.5, 9.5 and 17 kg.

Five trials per mass produce 4/5, 5/5, 5/5 and 3/5 successes with force adaptation, versus 4/5, 4/5, 2/5 and 1/5 without it. Both 9.5 and 17 kg exceed the simulation training mass limit. The 17-kg case demonstrates capability at 60% observed success, not a reliable-success guarantee.

IV-E Effectiveness of Visual Randomization

Visual augmentation improves real front/lateral/rear success counts from 8/10, 5/10 and 3/10 to 9/10, 8/10 and 8/10. The largest gains occur during longer repositioning, when viewpoint/background changes accumulate.

The comparison removes the combined augmentation strategy; it does not isolate the effect of each noise type. Observed confusion with background structures supports the proposed explanation, while ten trials per condition limit precision.

IV-F Closed-Loop Object Pushing

Three qualitative tests inspect feedback use beyond nominal straight pushing. They demonstrate selected successful responses rather than estimate recovery probabilities over standardized disturbance distributions.

External Perturbation.

A lateral impulse displaces the box. The robot stops forward motion, reorients to find it, navigates around it to restore object–goal alignment and resumes pushing. Figure 7a preserves the complete illustrated sequence.

Off-Centered Mass Distribution.

Weights attached to one side shift the center of mass, causing rotation and lateral deviation during pushing. Figure 1d shows the robot detecting drift and repositioning to continue toward the goal. Offset magnitudes and repeated-trial rates are not reported.

Sequential Goal Switching.

After reaching a front goal, supply a new goal behind the robot. It navigates around the same box and pushes toward the updated goal without reset. Figure 7b retains every frame; external goal updates are provided rather than autonomously selected.

V Conclusion, Limitations, and Future Work

The combination of privileged teacher learning, visual distillation, alignment shaping and force-adaptive control enables closed-loop pushing under varied goals and object masses.

Current evidence is limited to a single box-shaped object and relatively simple scenes. The policy assumes a relative goal position from another system, with motion capture used here. Multi-object clutter, more diverse background ambiguity and joint onboard object/goal localization remain open extensions.

Appendix: Teacher observation dimensions

Table III retains every actor/critic observation dimension. Actor entries sum to 56; adding the listed privileged critic entries gives 84. The printed entries include EE-object relative position 9, object rotation matrix 3, object mass 9 and object dimension 9.

Several labels/dimensions are unusual for the described physical quantities and may reflect reporting errors or unstated encodings. They remain unchanged because the paper does not resolve them; no corrected tensor shape is inferred.

Actor observations Dimensions
EE-object relative position 9
Object rotation matrix 3
Arm dof position 10
Arm dof velocity 10
Base linear velocity 3
Base angular velocity 3
Projected gravity 3
Relative goal position 2
Last action 13
Total dim 56
Priv. critic observations Dimensions
Object contact 1
Object CoM 3
Object mass 9
Object dimension 9
Object linear velocity 3
Object angular velocity 3
Total dim 84
TABLE III: Observation dimensions for an asymmetric actor-critic setup used to train the high-level teacher policy.

Appendix: Student observation dimensions

Table IV gives 67 proprioceptive dimensions: gravity 3, angular velocity 3, joint position/velocity 23 each, previous action 13 and goal position 2. Three depth frames at 32×32 form the image input. Temporal history encoding is additional to this per-step vector.

Student proprioception observations Dimensions
Projected gravity 3
Base angular velocity 3
Dof position 23
Dof velocity 23
Last action 13
Relative goal position 2
Total dim 67
Student image observations Dimensions
Depth image (Nimg×H×W)(N_{\text{img}}\times H\times W) 3×32×323\times 32\times 32
TABLE IV: Observation dimensions for the high-level student policy.

Appendix: Reward specification

Table V specifies (σ, weight) as reach (2.0, 1.25), push (10.0, 2.5), head tracking (0.5, 1.0) and alignment (0.25, 2.0). Penalty weights are −1.0 for command smoothness and −0.5 each for object tilt and upper-joint action magnitude.

All original penalty expressions and values are preserved. The paper does not provide every PPO/DAgger optimizer setting, exact normalized-angle mapping or full episode termination specification.

Main Reward Param. σ\sigma Weight
Reach object σreach=2.0\sigma_{\text{reach}}=2.0 1.251.25
Push object to goal σpush=10.0\sigma_{\text{push}}=10.0 2.52.5
Head track object σtrack=0.5\sigma_{\text{track}}=0.5 1.01.0
Object-goal alignment σalign=0.25\sigma_{\text{align}}=0.25 2.02.0
Auxilary Penalty Expression Weight
Command smoothness ‖𝐮t−𝐮t−1‖22\|\mathbf{u}_{t}-\mathbf{u}_{t-1}\|_{2}^{2} −1.0-1.0
Object balance ‖θobject tilt‖22\|\theta_{\text{object tilt}}\|_{2}^{2} −0.5-0.5
Upper joint action ‖𝐪tupper‖22\|\mathbf{q}^{\text{upper}}_{t}\|_{2}^{2} −0.5-0.5
TABLE V: Reward terms and weights
Concept demonstration

Open image ↗