Literature
Display
← All papers

RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learning

Yichao Zhong · Yidan Lu · Yuhang Lu · Tianyang Tang · Haoguang Mai · Yixuan Pan · Tianyu Li · Li Chen · Jingbo Wang · Zhongyu Li · Peng Lu · Hongyang Li

Preprint posted · CoRL 2026 (accepted) ↗

Robotics12 min text · CondensedOriginal paper ↗
Contents

Overview

This edition covers the complete sixteen-page v3 dated 11 June 2026. It retains all six numbered figures, eight numbered tables, the full training/deployment appendix and the final list of proposed future diagnostics.

Original full-resolution teaser photograph and ghosted kick sequence.
Figure 1: RoboNaldo: accurate, stable and powerful whole-body soccer shooting on a real grassy soccer field. Ghost-overlaid time sequence of a Unitree G1 humanoid executing a directed penalty kick.

At a glance

RoboNaldo trains one low-level humanoid controller in three stages: imitate a human kick, adapt it to target-directed stationary-ball shooting, then combine locomotion commands and a kick trigger for moving balls. A heuristic high-level planner handles approach and timing; task rewards reshape contact while motion tracking supplies balance.

A Unitree G1 uses onboard LiDAR/camera sensing and a specially reflective ball. Among valid launches, real free-kick and moving-ball errors average .73 m and .86 m from 3 m. These figures exclude failed launches: contact rates are 91.2% and 74.1%, and moving-ball balance survival is 88.9%. Peak reported ball speed is 13.10 m/s.

1 Introduction

Shooting combines single-leg balance, brief foot–ball contact, aim and power. Fixed motion imitation provides coordination but cannot choose a new strike time or contact geometry; task-only learning must discover these behaviors with sparse delayed feedback.

RoboNaldo stages those difficulties. Its point-target and moving-ball evaluations complement prior humanoid soccer and PAiD systems. The contribution is the combined curriculum, timing interface and reward design, rather than a general autonomous soccer player.

2 Related Work

Related work motivates a distinction between controlling brief impulse contacts and sustaining force over many timesteps, and between goal-region success and point-target accuracy.

Humanoid soccer and the shooting gap.

Prior soccer controllers emphasize locomotion, reactive kicking or goal-region entry. The authors argue that cosine-direction metrics can hide substantial lateral error at distance. Table 1 retains their capability comparison, including concurrent systems; its check marks describe reported evaluation coverage rather than a common empirical benchmark.

Table 1: Comparison of different humanoid soccer systems. We compare target-point accuracy (Point-acc), reported ball speed (Speed), moving-ball shooting (Moving), egocentric sensing (Ego), and outdoor demonstration (Out). †\dagger denotes concurrent work.
Method Shooting capability Deployment
Point-acc Speed Moving Ego Out
STOFT [8] ×\times ×\times ×\times ×\times ×\times
Reactive [27] ×\times ×\times ×\times ✓ ✓
Striker [30] ×\times ×\times ×\times ✓ ×\times
HumanX† [26] ×\times ×\times ✓ ×\times ×\times
PAiD† [7] ×\times ×\times ✓ ✓ ✓
RoboNaldo(Ours) ✓ ✓ ✓ ✓ ✓

Humanoid whole-body object interaction.

Whole-body interaction methods such as HDMI and ResMimic provide useful motion/contact machinery, but shooting gives a much shorter useful contact interval. The new reward preserves approach and outcome feedback around the impact instead of requiring sustained simultaneous position/force alignment.

Curriculum reinforcement learning.

The curriculum jointly changes task difficulty and learning signals. Ball diversity is introduced after a reliable kick exists, followed by explicit moving-ball timing. This follows the broader curriculum-RL idea of ordering learning problems, with task-specific stage transitions.

3 Method

The method combines a low-level motion-conditioned policy, a sequential training schedule and task-specific rewards. The third stage exposes approach velocity/heading and kick triggering as high-level commands.

3.1 Task Formulation

Policy inputs use robot-frame ball and target information, while evaluation measures the launched ball relative to the target in world coordinates.

Soccer shooting task.

Free-kicks start with a stationary ball; moving-ball trials add incoming velocity and uncertain arrival time. Success uses the minimum ball–target distance after first valid foot contact. This is a closest-approach criterion, not inherently the same as goal-line placement or a scored goal.

Equation
mint≥tc⁡‖pb​(t)−pt‖2≤dthresh,\min_{t\geq t_{c}}\|p_{b}(t)-p_{t}\|_{2}\leq d_{\rm thresh},

Policy observation.

The actor receives reference-joint cues, proprioception and prior actions with five-step histories, plus ball and target positions. Stage 3 replaces the reference-anchor cue with a nine-dimensional command while preserving the 547-dimensional input. A trigger changes the reference phase from locomotion to kicking; the critic has privileged noise-free body state.

Complete original figure with every panel, label and legend.
Figure 2: RoboNaldo motion-guided curriculum. A single low-level policy is trained sequentially across three stages. Stage 1 learns a stable kick by tracking a retargeted human motion. Stage 2 introduces shooting task rewards to adapt the kick for accurate stationary-ball free-kicks. Stage 3 replaces the reference motion and anchor cue with high-level locomotion and kick-trigger decision commands, enabling the policy to approach, trigger, strike, and stabilize when shooting moving balls.

3.2 Curriculum Design

Stage 1 retargets one side-foot human kick using GVHMR and GMR, then learns motion tracking following BeyondMimic without ball-task rewards. Stage 2 introduces the ball, target and spawn diversity so the strike can adapt to free-kicks. Stage 3 adds moving balls and separates locomotion, kick and post-kick stabilization.

The heuristic planner extrapolates planar ball position, guides the robot toward a feasible approach and triggers when predicted closest distance and height satisfy reachability gates. It snaps the motion clock to the kick phase with training jitter, then holds a stabilization reference after contact.

Stages advance using observed behavior, reward plateaus and converged action-noise scale. These are practical judgments rather than a fully specified automatic promotion threshold. The previous checkpoint initializes each next stage.

Equations
𝐩~​b=𝐩​b,⟂+𝐯​b,⟂Th\tilde{\mathbf{p}}b{=}\mathbf{p}{b,\perp}{+}\mathbf{v}{b,\perp}T_{h}
(dmin<rthr)∧(zb<hmax)∧(ϕ<ϕkick)(d_{\min}<r_{\rm thr})\wedge(z_{b}<h_{\max})\wedge(\phi<\phi_{\rm kick})

High-level control interface.

The command/trigger interface separates high-level decisions from the low-level shooting policy. Other controllers can in principle supply the same inputs without retraining the low-level model; the demonstrated system uses the heuristic planner, and a broadly evaluated learned high-level replacement is future work.

3.3 Reward Design

The total reward combines motion tracking, shooting task terms and regularization. Appendix B retains all component definitions, weights and stage scales; shaping accuracy and maintaining stable contact must be balanced.

Equation
r=rmotion​wmotion+rtask​wtask+rreg​wregr=r_{\text{motion}}w_{\text{motion}}+r_{\text{task}}w_{\text{task}}+r_{\text{reg}}w_{\text{reg}}

Motion tracking reward.

Exponential tracking terms cover fourteen body parts and emphasize foot velocity. They provide coordinated balance and swing structure, with distance-dependent relaxation near the ball in Stage 3.

Regularization reward.

Regularizers constrain action changes, second differences, slip, airborne behavior, joint speed/torque and limit violations. The published sign/weight conventions are retained exactly; the table sometimes prints both a negative quantity and a negative weight without resolving how these are combined in implementation.

Soccer shooting task reward.

Task shaping rewards approach, contact alignment, speed and eventual placement. A sparse target threshold is .5 m in Stage 2 and .35 m in Stage 3; a dense exponential term uses the best ball–target distance so far.

An additional ballistic extrapolation predicts where the current post-contact trajectory crosses the goal line, giving earlier placement feedback. It is a simplified shaping model with a floor clamp, not a full friction/bounce model or a replacement for the reported evaluation metric.

Equations
rsparse goal=𝟏[d^≤dthreshold]r_{\text{sparse goal}}{=}\mathbf{1}[\hat{d}\leq d_{\text{threshold}}]
rdense goal=exp(−d^2/σgoal2)r_{\text{dense goal}}{=}\exp(-\hat{d}^{2}/\sigma_{\text{goal}}^{2})
d^=mint′≤t⁡‖𝐩ball−𝐩target‖\hat{d}{=}\min_{t^{\prime}\leq t}\|\mathbf{p}_{\text{ball}}-\mathbf{p}_{\text{target}}\|

Instant interaction reward.

The interaction reward multiplies a sum of approach/outcome terms by a sum of velocity/force terms. Adding within each group keeps feedback available around a brief impact instead of making it vanish whenever one contact signal is absent.

The source calls shooting contact sub-10 ms in related work, but later describes three to five physics steps at 200 Hz, which is 15–25 ms. Both statements are retained as a source timing inconsistency.

Equation 1
rinteract=(Rcontact+Rgoal)​(Rvel+Rforce)/4,r_{\text{interact}}=\bigl(R_{\text{contact}}+R_{\text{goal}}\bigr)\bigl(R_{\text{vel}}+R_{\text{force}}\bigr)/4, (1)

4 Simulation Experiments

Simulation compares direct reinforcement learning, motion-prior approaches, a concurrent soccer baseline and staged RoboNaldo checkpoints, followed by curriculum and mechanism ablations.

Setup and metrics.

Training uses 4,096 parallel environments on an RTX 4090; evaluation uses 16,384 held-out episodes. Targets span an 8×2 m plane 5 m ahead. Stationary balls are sampled in a frontal square; moving balls approach at up to 5 m/s.

Metrics include nearest target error, .5/1 m success, post-contact peak ball speed, contact and survival. Table 2 reports ± values, but the text does not clearly specify independent training-seed count or the meaning of every uncertainty entry.

Experimental results.

Stage 2 free-kicks achieve .899 m error, 65.5% success within 1 m and 14.792 m/s peak-speed metric, compared with PAiD’s 1.850 m, 19.5% and 4.986 m/s. Stage 3 retains strong free-kicks and reaches 63.3% within 1 m on moving balls, with 79.2% contact and 98.8% survival.

Directly applying Stage 2 to moving balls gives only 7.2% within 1 m and 15.3% contact. Pure tracking is stable but poorly directed; task-only PPO and AMP struggle. These are the authors’ implementations and evaluation settings, not universal rankings of the algorithms.

The printed .899/1.850 error ratio is 48.6%, corresponding to about 51.4% lower error. The abstract instead calls it 48.6% lower; the numeric table is retained as the comparison basis.

Table 2: Simulation results and ablations. Shot err. is nearest ball–target distance; suc. denotes success rates with shot error less than 0.5/1.0  m; vballm​a​xv_{\mathrm{ball}}^{max} is peak post-contact ball speed. Free-kick uses a stationary ball; moving-ball shooting redirects a ground ball launched at 0–5 m/s. Contact denotes the ratio of envs where the robot contacts with ball. Ablated variants warm-start from the corresponding RoboNaldo stage checkpoint. Nominal policies and metrics are bold.
Entry Regime 0.5m suc. (%) ↑\uparrow 1.0m suc. (%) ↑\uparrow Shot err. (m) ↓\downarrow vballm​a​xv^{max}_{\text{ball}} (m/s) ↑\uparrow Contact (%) ↑\uparrow Alive (%) ↑\uparrow
Main comparison: free-kick
   PPO [19] Free-kick 0.0±0.00.0\pm 0.0 0.0±0.00.0\pm 0.0 4.721±0.0074.721\pm 0.007 0.780±0.0030.780\pm 0.003 0.0±0.00.0\pm 0.0 0.0±0.00.0\pm 0.0
   AMP [16] Free-kick 0.9±0.10.9\pm 0.1 3.3±0.13.3\pm 0.1 3.733±0.0043.733\pm 0.004 1.771±0.0511.771\pm 0.051 38.6±2.838.6\pm 2.8 41.6±2.341.6\pm 2.3
   PAiD [7] Free-kick 8.2±0.78.2\pm 0.7 19.5±0.519.5\pm 0.5 1.850±0.0261.850\pm 0.026 4.986±0.0174.986\pm 0.017 67.7±0.367.7\pm 0.3 67.3±0.567.3\pm 0.5
   Stage 1 motion tracking Free-kick 2.6±0.12.6\pm 0.1 9.2±0.39.2\pm 0.3 2.835±0.0082.835\pm 0.008 1.353±0.0061.353\pm 0.006 27.9±0.627.9\pm 0.6 99.9±0.199.9\pm 0.1
   RoboNaldo stage 2 Free-kick 28.8±1.328.8\pm 1.3 65.5±0.665.5\pm 0.6 0.899±0.0120.899\pm 0.012 14.792±0.01114.792\pm 0.011 82.1±0.382.1\pm 0.3 100.0±0.0100.0\pm 0.0
   RoboNaldo stage 3 Free-kick 36.3±0.136.3\pm 0.1 66.5±0.266.5\pm 0.2 1.055±0.0121.055\pm 0.012 14.178±0.03414.178\pm 0.034 81.1±0.181.1\pm 0.1 99.7±0.199.7\pm 0.1
Main comparison: moving-ball shooting
   PPO [19] Moving 0.6±0.10.6\pm 0.1 2.3±0.12.3\pm 0.1 3.962±0.0043.962\pm 0.004 4.251±0.0134.251\pm 0.013 0.5±0.10.5\pm 0.1 0.0±0.00.0\pm 0.0
   AMP [16] Moving 0.9±0.10.9\pm 0.1 3.5±0.13.5\pm 0.1 3.575±0.0153.575\pm 0.015 4.334±0.0124.334\pm 0.012 12.5±0.212.5\pm 0.2 15.7±0.315.7\pm 0.3
   PAiD [7] Moving 3.5±0.43.5\pm 0.4 9.3±0.69.3\pm 0.6 2.149±0.0342.149\pm 0.034 5.030±0.0055.030\pm 0.005 50.6±0.650.6\pm 0.6 48.1±0.548.1\pm 0.5
   Stage 2 zero-shot Moving 1.9±0.11.9\pm 0.1 7.2±0.17.2\pm 0.1 3.159±0.0033.159\pm 0.003 4.250±0.0094.250\pm 0.009 15.3±0.415.3\pm 0.4 37.4±0.237.4\pm 0.2
   RoboNaldo stage 3 Moving 32.4±0.632.4\pm 0.6 63.3±0.663.3\pm 0.6 1.131±0.0041.131\pm 0.004 13.875±0.00413.875\pm 0.004 79.2±0.179.2\pm 0.1 98.8±0.198.8\pm 0.1
Ablations: curriculum
   Stage 0→\!\to\!2 Free-kick 0.0±0.00.0\pm 0.0 0.3±0.10.3\pm 0.1 4.302±0.0164.302\pm 0.016 4.586±0.2104.586\pm 0.210 18.8±0.518.8\pm 0.5 1.2±0.11.2\pm 0.1
   Stage 1→\!\to\!3 Moving 1.7±0.21.7\pm 0.2 6.4±0.26.4\pm 0.2 3.453±0.0333.453\pm 0.033 4.196±0.0054.196\pm 0.005 13.9±0.813.9\pm 0.8 65.3±0.565.3\pm 0.5
   Stage 2→\!\to\!3 w/o planner Moving 0.9±0.30.9\pm 0.3 3.9±0.33.9\pm 0.3 3.283±0.0243.283\pm 0.024 4.710±0.0064.710\pm 0.006 19.1±1.119.1\pm 1.1 18.9±1.018.9\pm 1.0
Ablations: mechanism
   W/o adaptive sampling Moving 4.5±0.24.5\pm 0.2 12.6±0.312.6\pm 0.3 3.398±0.0113.398\pm 0.011 12.084±0.06412.084\pm 0.064 28.9±0.228.9\pm 0.2 32.8±2.032.8\pm 2.0
   W/o stabilization Moving 15.3±0.215.3\pm 0.2 25.6±0.325.6\pm 0.3 3.107±0.0363.107\pm 0.036 12.967±0.02312.967\pm 0.023 34.1±1.034.1\pm 1.0 24.4±1.224.4\pm 1.2
Ablations: interaction reward
   HDMI-style reward Free-kick 2.5±0.12.5\pm 0.1 8.9±0.28.9\pm 0.2 2.865±0.0012.865\pm 0.001 2.715±0.0122.715\pm 0.012 45.0±0.445.0\pm 0.4 98.7±0.298.7\pm 0.2

Ablations.

Skipping motion pretraining, skipping free-kick adaptation or omitting the Stage 3 planner each causes large performance losses. Removing adaptive sampling or post-kick stabilization reduces moving-ball survival to 32.8% or 24.4%, despite high ball speed.

Replacing the interaction reward with the HDMI-style alternative lowers free-kick .5 m success from 28.8% to 2.5%. The variants warm-start from relevant stage checkpoints. A separate weight-only/difficulty-controlled ablation is planned rather than completed.

5 Real-World Experiments

Hardware experiments exercise onboard sensing and control indoors and outdoors. The quantitative accuracy table separates launched shots from all attempts.

5.1 Hardware Setup and Perception

The 35 kg, 29-DoF Unitree G1 uses a size-5 ball, a head-mounted Livox MID-360 and chest RealSense D435. ONNX inference runs onboard at 50 Hz over DDS; no external motion-capture state estimate is used during deployment.

The ball is retro-reflective, and an AprilTag board or fixed coordinate specifies the target. The sensing arrangement is therefore instrumented; ordinary unmodified soccer-ball perception is not established.

Complete original figure with every panel, label and legend.
Figure 3: Real-world perception stack. A head-mounted MID-360 LiDAR tracks the retro-reflective ball at close range, while a chest-mounted IR camera extends ball detection beyond the LiDAR near-field. Both estimates are fused by a constant-velocity Kalman filter and transformed into the pelvis frame for policy input. The target board is localized by AprilTags from the same chest camera.
Complete original figure with every panel, label and legend.
Figure 4: Real-robot shooting overview. A: front-goal view shot dispersion; B: free-kick shot trajectories; C: per-region accuracy success rates and speed; D: moving-ball shot trajectories.

5.2 Experimental Results

Of 136 free-kick attempts, 124 launch the ball and all remain upright. On those valid launches, .5/1 m success is 31.5/80.6%, mean error .73 m and mean speed 7.42 m/s. Center targets are most accurate (.65 m), while left shots are faster and right shots slower, consistent with a right-footed bias.

Of 27 human-passed moving-ball attempts, 20 launch successfully. Conditional success is 30/70%, error .86 m and mean speed 7.10 m/s; overall contact and survival are 74.1% and 88.9%. Missed launches and falls must therefore be considered separately from the accuracy of successful kicks.

The maximum observed speed, 13.10 m/s, is distinct from the mean speeds. Its comparison with published professional-player speeds is contextual and uses different settings and summary statistics, not a matched human-versus-robot experiment.

Figure 4C’s regional success percentages match denominators using all attempts, while Table 3 explicitly conditions accuracy on valid launches. For example, the left region shows about 26/66% in the plot versus 30.6/77.6% in the table; these answer different reliability questions.

Complete original figure with every panel, label and legend.
Figure 5: Real-robot shooting showcase. (a) outdoor free-kick and (b) moving-ball one-touch shooting are time-lapses read left-to-right. (c)–(f) are four free-kick trials read top-to-bottom (contact frame above, landing outcome below), with landing errors of 31.531.5, 38.738.7, 45.345.3, and 35.935.9 cm.
Table 3: Real-world shooting results. Metrics follow Table 2; moving-ball targets are random.
ntotaln_{\text{total}} nvalidn_{\text{valid}} 0.5m suc. (%) ↑\uparrow 1.0m suc. (%) ↑\uparrow Shot err. (m) ↓\downarrow vballv_{\text{ball}} (m/s) ↑\uparrow Contact (%) ↑\uparrow Alive (%) ↑\uparrow
Free-kick
   Overall 136 124 31.5 80.6 0.73±0.390.73\pm 0.39 7.42±2.627.42\pm 2.62 91.2 100.0
    Left 58 49 30.6 77.6 0.74±0.330.74\pm 0.33 7.97±2.987.97\pm 2.98 84.5 100.0
    Center 50 48 35.4 85.4 0.65±0.340.65\pm 0.34 7.47±2.307.47\pm 2.30 96.0 100.0
    Right 28 27 25.9 77.8 0.85±0.520.85\pm 0.52 6.35±2.066.35\pm 2.06 96.4 100.0
Moving-ball 27 20 30.0 70.0 0.86±0.470.86\pm 0.47 7.10±2.407.10\pm 2.40 74.1 88.9

6 Discussion and Conclusion

The staged combination produces directed, high-speed kicks on the tested humanoid with onboard sensing. Explicit timing and post-impact stabilization are central to moving-ball performance, whose contact reliability remains below stationary free-kicks.

Limitations and future work.

The system depends on one reference motion, a handcrafted high-level trigger and a retro-reflective ball. Multiple skills, autonomous soccer decisions, learned timing and robust natural-ball perception are future directions rather than demonstrated capabilities.

Acknowledgments

The work acknowledges support from the Hong Kong Jockey Club Charities Trust’s JC STEM laboratory and the University of Hong Kong’s data-science seed fund, alongside named contributors to experiments and logistics.

Appendix A Observation Space: Per-Dimension Breakdown

Table A.1 preserves the full 547-dimensional observation vector. Reference joints contribute 58 dimensions, anchor/command cues nine, proprioceptive histories 450 and ball/target histories thirty. Stage 3 changes the nine-dimensional cue without changing actor input size; the critic adds privileged body state.

Table A.1: Policy observation vector (547 dims). †In Stage 3 the anchor reference is replaced by a 9-dim heuristic locomotion command (target body velocity/heading), keeping the total at 547. The critic additionally receives privileged, noise-free ground-truth body positions/orientations/velocities.
Group Component Dim.
Motion reference reference joint positions, 𝐪ref\mathbf{q}_{\text{ref}} (29 DOF) 29
reference joint velocities, 𝐪˙ref\dot{\mathbf{q}}_{\text{ref}} (29 DOF) 29
Anchor reference† anchor (torso) position in body frame, 𝐩anc\mathbf{p}_{\text{anc}} 3
anchor orientation (6D rotation), 𝐑anc\mathbf{R}_{\text{anc}} 6
Proprioception (×\times5 history) base angular velocity 15
joint positions (29 DOF) 145
joint velocities (29 DOF) 145
previous actions (29 DOF) 145
Exteroception (×\times5 history) ball position in robot frame 15
target position in robot frame 15
Total 547

Appendix B Reward Terms and Weights

Table B.1 specifies tracking, task and regularization terms, while Table B.2 gives outer curriculum weights and spawn settings. Tracking kernels use base scales divided by two in Stages 2–3, and Stage 3 additionally relaxes their weights near the ball.

The appendix prose says regularization scale .2 in Stages 1–2, whereas Table B.2 gives Stage 1 outer regularization weight zero. The nested task/component scales and the published discrepancy are retained; they should not be collapsed into an assumed single weight.

Table B.1: Reward terms. Motion-tracking terms use an exp. kernel exp(−∥𝐞∥2/σ2)\exp(-\|\mathbf{e}\|^{2}/\sigma^{2}); σ0\sigma_{0} is the base std, with σ=σ0/α\sigma{=}\sigma_{0}/\alpha (α=2\alpha{=}2) in Stages 2–3 and a proximity-based relaxation factor gi​(dball)g_{i}(d_{\text{ball}}) in Stage 3 (Table B.3). wgw_{g}: goal weight (00/11); wrw_{r}: regularization weight (0.20.2/0.250.25); per-stage scale shown as St. 1–2 / St. 3.
Term Quantity / form Weight
Motion tracking
anchor position (σ0=0.3\sigma_{0}{=}0.3 m) torso position vs. ref. 0.10.1
anchor orientation (σ0=0.4\sigma_{0}{=}0.4 rad) torso orientation vs. ref. 0.50.5
body positions (σ0=0.3\sigma_{0}{=}0.3 m) 14 relative body positions 1.01.0
body orientations (σ0=0.4\sigma_{0}{=}0.4 rad) 14 body orientations 1.01.0
body linear vel. (σ0=1.0\sigma_{0}{=}1.0 m/s) 14 body linear velocities 1.01.0
body angular vel. (σ0=π\sigma_{0}{=}\pi rad/s) 14 body angular velocities 1.01.0
feet linear vel. (σ0=1.0\sigma_{0}{=}1.0 m/s) 2 feet linear velocities 1.01.0
Task (×wg\times w_{g})
error_ball_to_target exp; mint⁡‖𝐩b−𝐩t‖\min_{t}\|\mathbf{p}_{b}{-}\mathbf{p}_{t}\| (σ=0.5/α\sigma{=}0.5/\alpha); =1=1 within thr. 5​wg5w_{g}
predicted_error_ball_to_target exp; ‖𝐩^−𝐩t‖\|\hat{\mathbf{p}}{-}\mathbf{p}_{t}\| (Eq. 3, σ=1/α\sigma{=}1/\alpha) 10​wg10w_{g}
goal_reward_burst constant for 10 steps on success (delay 100) 300​wg300w_{g}
robot_ball_contact ankle–𝐩exec\mathbf{p}_{\text{exec}} proximity ++ contact force, buffered 2​wg2w_{g}
contact orientation ankle velocity vs. ball→\totarget 1​wg1w_{g}
ankle–ball / torso–ball distance dballd_{\text{ball}} / torso–ball dist. 1​wg1w_{g} / 1.5​wg1.5w_{g}
ball_velocity sat. Lorentzian on ‖𝐯b‖\|\mathbf{v}_{b}\| post-contact 0.5​wg0.5w_{g}
alive constant while upright 0.5​wg0.5w_{g}
Regularization (×wr\times w_{r})
action rate −‖Δ​𝐚‖2-\|\Delta\mathbf{a}\|^{2} −0.1-0.1 (1.0/1.5)
action smoothness −‖Δ2​𝐚‖2-\|\Delta^{2}\mathbf{a}\|^{2} −10−5-10^{-5} (1.0/2.0)
feet slip −‖𝐯foot‖2-\|\mathbf{v}_{\text{foot}}\|^{2} in stance −10−2-10^{-2}
no-fly −𝟙​[both feet airborne]-\mathbb{1}[\text{both feet airborne}] −0.2-0.2 (1.0/1.25)
joint velocity −‖𝐪˙‖2-\|\dot{\mathbf{q}}\|^{2} −10−5-10^{-5}
joint torque −‖𝝉‖2-\|\boldsymbol{\tau}\|^{2} −2×10−7-2{\times}10^{-7}
joint limit (always active) −𝟙​[violation]-\mathbb{1}[\text{violation}] −10-10 (1.0/1.5)
undesired contacts contact off ankles/wrists −0.1-0.1
feet contact time smooth foot-contact duration −0.05-0.05 (1.0/1.2)
Table B.2: Three-stage curriculum. Ball spawns relative to (1.0,−0.2,0.115)(1.0,-0.2,0.115) m; †kick/loco mode. Full observation vector in Table A.1.
S1 S2 S3
Tracking Adapt Generalize
wtaskw_{\text{task}} 0.0 0.8 1.0
wregw_{\text{reg}} 0.0 0.2 0.25
wmotionw_{\text{motion}} 1.0 1.0 1.0 / 0.1†
Ball spawn XY (m) — ±\pm0.75 ±\pm1.0
Ball vel. XY (m/s) 0 0.1 m/s 5 m/s
EEF Δ​pz\Delta p_{z} term. (m) [9] 0.25 0.5 0.5

Instant Interaction Reward factors.

Equation 2 expands the interaction factors: foot-to-ball clearance shapes approach, best-so-far target distance shapes outcome, a thresholded speed term rewards useful velocity and a capped force term rewards impact. The original clipping, thresholds and kernel powers remain visible.

Equation 2
Rcontact\displaystyle R_{\text{contact}} =exp⁡(−[‖𝐩foot−𝐩ball‖−rb]+σf),\displaystyle=\exp\!\Bigl(-\tfrac{[\|\mathbf{p}_{\text{foot}}{-}\mathbf{p}_{\text{ball}}\|-r_{b}]_{+}}{\sigma_{f}}\Bigr), Rgoal\displaystyle R_{\text{goal}} =exp⁡(−[d^−σg2]+2(2​σg)2),\displaystyle=\exp\!\Bigl(-\tfrac{[\hat{d}-\frac{\sigma_{g}}{2}]_{+}^{2}}{(2\sigma_{g})^{2}}\Bigr), (2)
Rvel\displaystyle R_{\text{vel}} =1−exp⁡(−[‖𝐯ball‖−vmin]+210),\displaystyle=1-\exp\!\Bigl(-\tfrac{[\|\mathbf{v}_{\text{ball}}\|-v_{\text{min}}]_{+}^{2}}{10}\Bigr), Rforce\displaystyle R_{\text{force}} =min⁡(ffthresh,1),\displaystyle=\min\!\Bigl(\tfrac{f}{f_{\text{thresh}}},1\Bigr),

Densified Shooting Reward.

Equation 3 estimates time to the target’s goal-line coordinate using ball velocity, then predicts position under gravity. Positive predicted time activates the dense target-error reward; height is clamped at ball radius, and the source switches to actual landing position after crossing. This provides earlier feedback under an approximate trajectory model.

Equation 3
t∗\displaystyle t^{*} =pt,y−pb,yvb,y+ϵ,𝐩^=𝐩b+𝐯b​t∗−12​g​(t∗)2​𝐳^,\displaystyle=\frac{p_{t,y}-p_{b,y}}{v_{b,y}+\epsilon},\qquad\hat{\mathbf{p}}=\mathbf{p}_{b}+\mathbf{v}_{b}\,t^{*}-\tfrac{1}{2}g\,(t^{*})^{2}\hat{\mathbf{z}}, (3)
rdensified\displaystyle r_{\text{densified}} ={exp(−∥𝐩^−𝐩t∥2/σ2)t∗>00else,\displaystyle=\begin{cases}\exp\!\bigl(-\|\hat{\mathbf{p}}-\mathbf{p}_{t}\|^{2}/\sigma^{2}\bigr)&t^{*}>0\\[2.0pt] 0&\text{else,}\end{cases}

Proximity-based tracking relaxation.

Tracking weights interpolate from term-specific near-ball scales at .35 m to full weight at 1 m. Foot-velocity tracking falls to .05 while orientation retains .6, allowing contact adaptation without discarding balance guidance. Table B.3 preserves all seven near-ball factors; sensitivity to these choices remains proposed work.

Equation 4
gi​(d)=μi+(1−μi)⋅clamp⁡(d−dneardfar−dnear, 0, 1),dnear=0.35​ m,dfar=1.0​ m,g_{i}(d)\;=\;\mu_{i}+(1-\mu_{i})\cdot\mathrm{clamp}\!\left(\tfrac{d-d_{\text{near}}}{d_{\text{far}}-d_{\text{near}}},\,0,\,1\right),\qquad d_{\text{near}}=0.35\text{\,m},\;\;d_{\text{far}}=1.0\text{\,m}, (4)
Table B.3: Proximity-based tracking relaxation scales μi\mu_{i} (Stage 3, dball≤dnear=0.35d_{\text{ball}}\!\leq\!d_{\text{near}}{=}0.35 m).
Term anchor pos. anchor ori. body pos. body ori. body lin. vel. body ang. vel. feet lin. vel.
μi\mu_{i} 0.10 0.60 0.20 0.60 0.20 0.30 0.05

Appendix C Hyperparameters and Domain Randomization

The appendix specifies optimization, planner geometry, simulator rates and domain randomization. These details define the evaluated configuration, not a comprehensive parameter-sensitivity study.

PPO (RSL-RL).

PPO uses three-layer actor/critic MLPs of 512–256–128 units with ELU and observation normalization. Rollouts have 24 environment steps; updates use five epochs and four minibatches. The clip ratio is .2, discount .99 and GAE .95; adaptive learning starts at 2×10⁻⁵ with KL target .01.

The source gives clipped value loss with coefficient one, entropy .001, gradient clip one, initial action-noise standard deviation one, up to 100,000 iterations per stage and checkpointing every 1,000. These are configured limits; actual per-stage training duration and total compute are not reported.

Equations
2×10−52\times 10^{-5}
γ=0.99\gamma{=}0.99
λ=0.95\lambda{=}0.95
512→256→128512{\to}256{\to}128

Heuristic planner.

The heuristic predicts .4 s ahead, permits .25 m closest approach below .75 m height, and uses a nominal foot offset (.15,−.1) m. Kick-boundary frame 240 is sampled with ±10-frame jitter. Exact desired-position/orientation and closest-approach formulas remain below; the heuristic is specific to the chosen motion and embodiment.

Equations
𝐩des=𝐩anc,⟂+12​min⁡(‖𝐩~b−𝐩anc,⟂‖,2)​𝐝^b\mathbf{p}_{\text{des}}=\mathbf{p}_{\text{anc},\perp}+\tfrac{1}{2}\min(\|\tilde{\mathbf{p}}_{b}-\mathbf{p}_{\text{anc},\perp}\|,2)\hat{\mathbf{d}}_{b}
𝐪des=norm⁡([1+𝐞x⋅𝐝^b,𝐞x×𝐝^b])\mathbf{q}_{\text{des}}=\mathrm{norm}([1+\mathbf{e}_{x}{\cdot}\hat{\mathbf{d}}_{b},\mathbf{e}_{x}{\times}\hat{\mathbf{d}}_{b}])
𝐩~b=𝐩b,⟂+𝐯b,⟂​Th\tilde{\mathbf{p}}_{b}{=}\mathbf{p}_{b,\perp}{+}\mathbf{v}_{b,\perp}T_{h}
dmin=‖Δ​𝐩+t∗​𝐯b,⟂body‖d_{\text{min}}=\|\Delta\mathbf{p}+t^{*}\mathbf{v}_{b,\perp}^{\text{body}}\|
Δ​𝐩=𝐩b,⟂body−𝐩offset\Delta\mathbf{p}=\mathbf{p}_{b,\perp}^{\text{body}}-\mathbf{p}_{\text{offset}}
t∗=clamp(−Δ𝐩⋅𝐯b,⟂body/∥𝐯b,⟂body∥2,0,Th)t^{*}=\mathrm{clamp}(-\Delta\mathbf{p}{\cdot}\mathbf{v}_{b,\perp}^{\text{body}}/\|\mathbf{v}_{b,\perp}^{\text{body}}\|^{2},0,T_{h})

Simulation.

Physics runs at 200 Hz with fourfold decimation to 50 Hz control. Ten-second episodes contain 500 control steps. The simulated ball has .115 m radius, about .41 kg mass and .95 restitution; foot and ball contact sensors provide interaction feedback. Solver settings are eight position and four velocity iterations.

Domain randomization.

Per-episode robot static friction is sampled from .3–1.6, dynamic friction from .3–1.2 and restitution from 0–.5. Default joint positions vary by ±.01 rad; torso center of mass varies by ±.025 m fore–aft and ±.05 m laterally/vertically. Actuator delay is described only as a few control steps.

Every 1–3 s, random pushes add up to ±.5 m/s horizontal and ±.2 m/s vertical velocity, plus ±.52 rad/s roll/pitch and ±.78 rad/s yaw velocity. Ball spawn and initial velocity follow Table B.2. These are configured training disturbances, not measured deployment robustness limits.

Equations
𝒰⁡[0.3,1.6]\mathcal{U}[0.3,1.6]
𝒰⁡[0.3,1.2]\mathcal{U}[0.3,1.2]
𝒰⁡[0,0.5]\mathcal{U}[0,0.5]

Appendix D Deployment Details

Table D.1 compares sensor options. Its nominal LiDAR range and latency entries should be distinguished from this mounting’s approximately two-metre useful ground coverage and the roughly 10 Hz processed point-cloud stream described below.

Table D.1: Perception modules. Fast ball: robust to 15 m/s motion blur. Onboard: no external infra.
Module Range Lat. (ms) Fast ball Onboard
MoCap (Vicon) room <<5 ✓ ×\times
RGB (D435 color) 6 m 30–60 ×\times ✓
Depth (D435 depth) 3 m 30–60 – ✓
LiDAR (MID-360) 40 m 10–20 ✓ ✓

Perception.

Near-range ball localization filters LiDAR returns by reflectivity, range and region, then fits a known-radius sphere and smooths its center with a constant-velocity Kalman filter. Farther away, the D435 infrared stream supplies a bright-blob/circularity estimate fused into the same pelvis-frame state.

The source reports failures of RGB YOLO and color/background detectors under blur, lighting and clutter, without a quantified localization benchmark. AprilTags localize the goal board, or a fixed target coordinate is used; measurements older than 300 ms are invalidated. Reliable long-range tracking of a fast ball remains future work.

Joint-index remapping.

Isaac Lab and Unitree/MuJoCo order joints differently. A fixed permutation and inverse map observations and actions between the conventions. The appendix says this map is in released code, but does not print its entries here; this edition does not invent them or infer release availability.

Initialization and control loop.

Deployment zero-fills the five-step histories and linearly blends the initial configuration into the first policy action over ten control steps. The 50 Hz loop reads joint/IMU and ball/target states, performs ONNX inference and sends checked position commands with PD gains. A dedicated button switches to immediate damping; this is an operator fallback, not a formal safety guarantee.

Equation
𝐪target(t)=(1−t/10)​𝐪0+(t/10)​𝐚1\mathbf{q}_{\text{target}}^{(t)}=(1-t/10)\,\mathbf{q}_{0}+(t/10)\,\mathbf{a}_{1}

Appendix E Additional Simulation Results

The twelve heatmaps show target-dependent error, peak speed and .5/1 m success for Stage 2 free-kicks, Stage 3 free-kicks and Stage 3 moving balls. Extreme lateral and high targets are harder; the panels retain the complete spatial coverage and color scales.

Complete original figure with every panel, label and legend.
Figure E.1: Target-conditioned shooting heatmaps. Rows correspond to Stage 2 free-kick, Stage 3 free-kick, and Stage 3 moving-ball shooting. Each heatmap visualizes one shooting metric over goal-plane target locations: shot error, peak ball speed, 1.0 m success rate, or 0.5 m success rate.

Appendix F Additional Results

Despite the heading “Additional Results,” this section lists planned diagnostics: localization error, relaxation-scale sensitivity, joint coordination, hardware failure breakdown with confidence intervals, and a weight-only ablation. None of those additional quantitative findings is supplied in this version.

Concept demonstration

Open image ↗