Literature
Display
← All papers

Echo in the Steps: Learning Perceptive Humanoid Parkour with Gated Memory

Ming-Ju Lee · Zizhuo Wang · Shaoting Zhu · Haozhe Lou · Hang Zhao · Yiming Li

Preprint posted · CoRL 2026 (accepted) ↗

Robotics9 min text · CondensedOriginal paper ↗
Contents

Overview

The complete 17-page paper studies depth-guided parkour on a 29-degree-of-freedom Unitree G 1. Figure 1 retains all six selected hardware examples, including 20 cm footholds, a curved narrow support and a 2 m beam. These illustrate the tested apparatus rather than a general outdoor-locomotion guarantee.

Complete original Figure 1: Perceptive parkour over challenging terrains. Our method enables a Unitree G1 humanoid robot to perform stable and agile parkour over diverse terrains with sparse footholds and narrow support regions using only onboard depth observations. The resulting policy demonstrates (a) traversal across trapezoids with large height variations, (b) robust locomotion over discontinuous stepping boxes, (c) accurate foot placement on sparse footholds only 20 cm in diameter, (d) coordinated alternating steps across irregularly oriented wedges, (e) agile traversal at approximately 1 m/s over a 20 cm-wide curved support, and (f) stable traversal along a 2 m-long narrow beam.
Figure 1: Perceptive parkour over challenging terrains. Our method enables a Unitree G1 humanoid robot to perform stable and agile parkour over diverse terrains with sparse footholds and narrow support regions using only onboard depth observations. The resulting policy demonstrates (a) traversal across trapezoids with large height variations, (b) robust locomotion over discontinuous stepping boxes, (c) accurate foot placement on sparse footholds only 20 cm in diameter, (d) coordinated alternating steps across irregularly oriented wedges, (e) agile traversal at approximately 1 m/s over a 20 cm-wide curved support, and (f) stable traversal along a 2 m-long narrow beam.

At a glance

The policy combines current depth features with a short history, using a hand-designed frame-saliency prior, learned temporal weights and gated residual fusion. A separate alternation loss complements mirror symmetry to discourage synchronized two-leg hopping.

Simulation comparisons and small hardware trials support these components on five terrain families. The paper’s main and appendix ablation aggregates differ, and implementation details contain a stagnation-threshold discrepancy; the notes preserve those distinctions.

1 Introduction

Sparse supports can disappear from an egocentric camera before the robot places its foot. Purely reactive control loses that geometry, while explicit maps add reconstruction and localization work.

This method retains recent depth features and biases their use toward frames with stronger depth discontinuities. The resulting representation joins proprioceptive history in an end-to-end policy. Gait regularization addresses a separate problem: mirror-consistent actions can still collapse into synchronized hopping.

“Depth-only” refers to exteroception. The actor also receives robot state, previous actions and velocity commands.

2 Related Work

The related work connects map-based and image-conditioned traversal, temporal memory, and symmetry-aware policy learning. A short feature history offers a simpler alternative to building a complete terrain map.

The proposed memory is a finite-window fusion mechanism, rather than a recurrent map with indefinitely persistent terrain state. The symmetry objective is an inductive bias; its effect on actual contact timing is evaluated empirically.

3 Problem Formulation

The task traverses elevated discrete platforms. An asymmetric actor–critic policy is trained with PPO [30]; the critic receives additional clean state and base linear velocity. The actor outputs 29 joint-position targets, converted to torques by the retained PD-control equation.

Rewards combine velocity tracking, regularization, safety penalties and an adversarial motion prior. These are training objectives rather than formal safety guarantees. Episodes stop for falls, excessive tilt, boundary/timeout conditions and prolonged starting-platform residence.

The main text says stagnation beyond three seconds terminates an episode; the appendix configuration lists four seconds. Both original descriptions remain available.

The motion-style term follows AMP [26].

Complete original Figure 2: Overview of the framework. (a) In simulation, a saliency-guided temporal perception module fuses depth images with saliency priors, and the resulting visual features are combined with proprioceptive observations to predict actions. (b) The learned policy is deployed on a real humanoid using only onboard depth observations for perception. (c) The saliency map is derived from vertical depth differences to highlight critical foothold regions. (d) The symmetry regularization combines a mirror-consistency objective with an alternation loss to encourage coordinated stepping.
Figure 2: Overview of the framework. (a) In simulation, a saliency-guided temporal perception module fuses depth images with saliency priors, and the resulting visual features are combined with proprioceptive observations to predict actions. (b) The learned policy is deployed on a real humanoid using only onboard depth observations for perception. (c) The saliency map is derived from vertical depth differences to highlight critical foothold regions. (d) The symmetry regularization combines a mirror-consistency objective with an alternation loss to encourage coordinated stepping.
Equation 1
𝝉t=kp​(𝐚t−𝐪t)−kd​𝐪˙t.\boldsymbol{\tau}_{t}=k_{p}(\mathbf{a}_{t}-\mathbf{q}_{t})-k_{d}\dot{\mathbf{q}}_{t}. (1)

4 Method

A shared encoder produces current and historical depth latents. Saliency-guided temporal weighting and a learned scalar gate combine them before the policy head. Mirror consistency and an alternation penalty regularize the action policy.

4.1 Saliency Prior

The saliency map takes absolute vertical pixel-to-pixel depth differences, multiplied by relative image row v/H to emphasize lower image regions. Averaging the map yields one scalar prior per frame.

This is a heuristic for potentially useful foothold geometry. It is not a foothold segmentation, a calibrated uncertainty measure or an explicit spatial planner; camera motion and depth artifacts can also create discontinuities.

Equations 2–3
𝐑t​(u,v)=vH​|𝐃t​(u,v)−𝐃t​(u,v−1)|,\mathbf{R}_{t}(u,v)=\frac{v}{H}\left|\mathbf{D}_{t}(u,v)-\mathbf{D}_{t}(u,v-1)\right|, (2)
rt=1W​H​∑u=1W∑v=1H𝐑t​(u,v).r_{t}=\frac{1}{WH}\sum_{u=1}^{W}\sum_{v=1}^{H}\mathbf{R}_{t}(u,v). (3)

4.2 Temporal Gated Memory

Historical latents are scored by an MLP using their relative saliency, with an additional learned saliency scale and frame bias. Softmax produces temporal weights. A sigmoid of a learned scalar β controls how far the output moves from the current latent toward the weighted history.

Because the weights sum to one, Equation 7 is a blend of current features and weighted historical features. The gate itself is a learned scalar rather than an observation-dependent recurrent update. Selectivity also comes from the frame scores.

The implementation reports a four-frame depth stack; the text does not fully specify sampling intervals, normalization of the relative saliency, encoder widths or all initialization choices.

Equations 4–7
𝐳c=ℰθ(𝐃c),𝐳th=ℰθ(𝐃th),t=1,…,τ.\mathbf{z}^{c}=\mathcal{E}_{\theta}(\mathbf{D}^{c}),\qquad\mathbf{z}^{h}_{t}=\mathcal{E}_{\theta}(\mathbf{D}^{h}_{t}),\qquad t=1,\dots,\tau. (4)
r^t=Norm(rth−rc),t=1,…,τ,\hat{r}_{t}=\operatorname{Norm}(r^{h}_{t}-r^{c}),\qquad t=1,\dots,\tau, (5)
wt=Softmax(fψ([𝐳th,r^t])+αr^t+bt),t=1,…,τ,w_{t}=\operatorname{Softmax}\!\left(f_{\psi}\!\left([\mathbf{z}^{h}_{t},\hat{r}_{t}]\right)+\alpha\hat{r}_{t}+b_{t}\right),\qquad t=1,\dots,\tau, (6)
𝐳~=𝐳c+σ⁡(β)​∑t=1τwt​(𝐳th−𝐳c),\tilde{\mathbf{z}}=\mathbf{z}^{c}+\sigma(\beta)\,\sum_{t=1}^{\tau}w_{t}\left(\mathbf{z}^{h}_{t}-\mathbf{z}^{c}\right), (7)
Complete original Figure 3: Saliency-guided temporal perception module with saliency weighting and gated memory update.
Figure 3: Saliency-guided temporal perception module with saliency weighting and gated memory update.

4.3 Symmetry Regularization

Mirror consistency compares the policy output with the mirrored output from mirrored observations. The alternation loss penalizes positive cosine similarity between left/right leg-action subvectors when base speed exceeds a threshold.

This encourages different leg actions while retaining bilateral structure. It does not directly impose a foot-contact phase schedule. Figure 6 tests whether actual contacts become more alternating.

Equations 8–9
ℒs=𝔼𝐨​[‖πθ​(𝐨)−Ma​(πθ​(Mo​(𝐨)))‖22],\mathcal{L}_{\mathrm{s}}=\mathbb{E}_{\mathbf{o}}\left[\left\|\pi_{\theta}(\mathbf{o})-M_{a}\!\left(\pi_{\theta}(M_{o}(\mathbf{o}))\right)\right\|_{2}^{2}\right], (8)
ℒa=𝔼𝐨​[1(‖𝐯t‖2>v0)​[max⁡(0,cos⁡(𝐚legL,𝐚legR))]2],\mathcal{L}_{\mathrm{a}}=\mathbb{E}_{\mathbf{o}}\left[\mathbf{1}\!\left(\|\mathbf{v}_{t}\|_{2}>v_{0}\right)\Big[\max\!\big(0,\cos(\mathbf{a}^{L}_{\mathrm{leg}},\mathbf{a}^{R}_{\mathrm{leg}})\big)\Big]^{2}\right], (9)

5 Experiments

Experiments ask about simulated traversal, component contributions and hardware transfer. All result tables are retained, including failed ablations and differences between main-text and appendix aggregates.

Complete original Table 1: Comparison of our method with Hiking : simulation success rate and foothold accuracy.
Table 1: Comparison of our method with Hiking: simulation success rate and foothold accuracy.
Accessible table and original citation links
Terrain Type Success Rate (%) Foothold Accuracy (%)
Hiking Ours Hiking Ours
Boxes 79.4±2.879.4\pm 2.8 98.1±0.4\mathbf{98.1}\pm 0.4 88.0±0.288.0\pm 0.2 91.9±0.2\mathbf{91.9}\pm 0.2
Stakes 70.7±1.770.7\pm 1.7 88.1±1.9\mathbf{88.1}\pm 1.9 74.9±0.774.9\pm 0.7 85.0±0.2\mathbf{85.0}\pm 0.2
Beams 82.2±1.482.2\pm 1.4 91.8±0.3\mathbf{91.8}\pm 0.3 80.9±0.480.9\pm 0.4 87.2±0.1\mathbf{87.2}\pm 0.1
Wedges 83.5±1.083.5\pm 1.0 88.4±0.4\mathbf{88.4}\pm 0.4 85.4±0.385.4\pm 0.3 88.3±0.2\mathbf{88.3}\pm 0.2
Trapezoids 64.2±0.364.2\pm 0.3 97.1±0.7\mathbf{97.1}\pm 0.7 49.7±0.749.7\pm 0.7 65.5±0.5\mathbf{65.5}\pm 0.5

5.1 Simulation Experiments

Training uses Isaac Sim/Isaac Lab, 2,048 parallel G 1 agents and an RTX 4090. Five terrain families—boxes, stakes, beams, wedges and trapezoids—follow a 20×10 difficulty curriculum. These categories appear in training; performance across them is not an unseen-category evaluation.

The policy uses eight proprioceptive frames and a four-frame depth stack. The source reports 5,000 evaluation trials per experiment and three seeds, with means and standard deviations. It does not clearly state whether 5,000 trials is per seed or pooled.

Success means reaching the end without ground contact or falling. Foothold accuracy is the fraction of foot-contact samples within intended supports during inter-platform traversal, rather than the fraction of entire trials completed.

Training uses Isaac Lab [25].

Complete original Table 2: Ablation study. Row 1 uniformly averages frame latents; row 2 replaces averaging with MLP-predicted softmax weighting; row 3 incorporates the
saliency prior into the learned weighting; and row 4 further adds gated residual fusion. Results are averaged over five terrain categories.
Table 2: Ablation study. Row 1 uniformly averages frame latents; row 2 replaces averaging with MLP-predicted softmax weighting; row 3 incorporates the saliency prior into the learned weighting; and row 4 further adds gated residual fusion. Results are averaged over five terrain categories.
Accessible table and original citation links
Learnable
Weight
Saliency
Prior
Gated
Memory
SR (%) FA (%)
✗ ✗ ✗ 52.12 70.80
✓ ✗ ✗ 40.65 82.65
✓ ✓ ✗ 88.14 90.63
✓ ✓ ✓ 98.27 91.10
Complete original Table 3: Effect of Symmetry Regularization .
Table 3: Effect of Symmetry Regularization.
Accessible table and original citation links
Method SR (%) FA (%)
w/o sym 85.0±1.185.0\pm 1.1 86.0±0.186.0\pm 0.1
w/ sym 96.7±0.9\mathbf{96.7\pm 0.9} 91.8±0.1\mathbf{91.8\pm 0.1}

5.1.1 Main Results

Table 1 averages 92.7% success and 83.6% foothold accuracy, versus 76.0% and 75.8% for the matched-training Hiking baseline: improvements of 16.7 and 7.8 percentage points. Trapezoid foothold accuracy remains 65.5% despite 97.1% trial success.

Table 2 gives 52.12% success for uniform fusion, 40.65% for learned weights alone, 88.14% with saliency and 98.27% with gated fusion. Learned weighting alone reduces success despite improving contact accuracy. The saliency and gate changes are 47.49 and 10.13 percentage points, respectively.

On boxes, symmetry regularization raises success from 85.0% to 96.7% in Table 3. Figure 5 shows faster training with alternation loss. Figure 6 reports double-support ratio falling from 0.806 to 0.308, supporting a change in gait coordination.

These ablation numbers are not interchangeable with Table 1 or the appendix’s terrain-wise ablation. The source does not reconcile their different full-model averages.

The main baseline is Hiking in the Wild [45].

Complete original Figure 4: Overview of real-world experiments across diverse terrains . Snapshots from trials over three representative terrains: trapezoids (top), stakes (middle), and narrow beams (bottom). During traversal, the policy maintains stable, coordinated stepping and accurate foot placement.
Figure 4: Overview of real-world experiments across diverse terrains. Snapshots from trials over three representative terrains: trapezoids (top), stakes (middle), and narrow beams (bottom). During traversal, the policy maintains stable, coordinated stepping and accurate foot placement.
Complete original Figure 5: Training Efficiency.
Figure 5: Training Efficiency.
Complete original Figure 6: Validation of the alternation loss . (a) Mean foot-contact probabilities over a normalized gait cycle. (b) Double-support ratio with and without the alternation loss.
Figure 6: Validation of the alternation loss. (a) Mean foot-contact probabilities over a normalized gait cycle. (b) Double-support ratio with and without the alternation loss.

5.2 Real-world Experiments

Hardware inference runs at 50 Hz on a Jetson Orin NX under ROS 2. A RealSense D435i captures 480×270 depth at 60 Hz; frames are downsampled to 64×36 and cropped to a 32×18 center-bottom region.

Each terrain setting receives ten independent trials. This is a small evaluation with coarse ten-percentage-point increments, and no hardware baseline comparison or interval estimate is supplied.

5.2.1 Main Results

The real-world plot reports high success over the five terrain families, with selected traversal sequences in Figures 1,4 and 9. These show the policy balancing and adjusting foot placement on the indoor apparatus.

Ten trials per setting and selected videos provide evidence of transfer, while leaving uncertainty about longer deployments, unseen layouts, camera failures and broader environments.

Complete original Figure 7: Real-world success rate.
Figure 7: Real-world success rate.

6 Conclusion

Saliency-guided short-window features and gait regularization improve the tested humanoid policy. The evidence supports these components within the reported curricula and hardware settings.

The results do not establish general navigation reliability, indefinite terrain memory or safe traversal under arbitrary perception failures.

7 Limitation

Staggered supports can induce poor support-leg choices and cross-stepping. A locally reachable foothold can make the next transition difficult, motivating explicit sequence-level footstep planning.

The camera’s limited field of view also hides upcoming supports during sharp heading changes. Wider or actively controlled perception is proposed future work.

Acknowledgments

The authors thank reviewers, the area chair and Unitree Robotics. Ming-Ju Lee and Zizhuo Wang contributed equally; Hang Zhao and Yiming Li are corresponding authors.

Original project demonstrations

The authors’ project page supplies an overview, full presentation, six one-shot examples and five longer-horizon demonstrations. All 13 original videos and their original posters are preserved locally.

These selected demonstrations supplement the quantitative protocol. They do not provide an independent hardware success-rate estimate or cover all failure cases.

hero · echo hero
full · full video
experiments · one shot · box
experiments · one shot · stake
experiments · one shot · trapezoid
experiments · one shot · wedge
experiments · one shot · beam 1
experiments · one shot · beam 2
experiments · long horizon · box
experiments · long horizon · beam
experiments · long horizon · stake
experiments · long horizon · trapezoid
experiments · long horizon · wedge

Appendix A Additional Experiments and Analysis

Appendix A adds terrain-wise memory ablations, symmetry illustrations and complete five-terrain hardware snapshot galleries. All original panels and data are retained.

A.1 Ablation Study of Gated Memory

Table 4 expands the memory ablation by terrain. With full fusion, success ranges 98.61–99.96%, averaging about 99.57%, while foothold accuracy averages 92.24%. These differ from Table 2’s 98.27/91.10 and Table 1’s 92.7/83.6; the source gives no reconciliation.

Gated fusion improves success in every displayed terrain, but trapezoid foothold accuracy decreases slightly from 87.97% to 87.67%. The full table preserves every mean, standard deviation and original red/blue ranking.

Complete original Table 4: Terrain-wise ablation study of gated memory . Red and blue highlight the best and second-best results for each metric, and gray parentheses report standard deviations.
Table 4: Terrain-wise ablation study of gated memory. Red and blue highlight the best and second-best results for each metric, and gray parentheses report standard deviations.
Accessible table and original citation links

Learnable Weight Saliency Prior Gated Memory Boxes Stakes Beams Wedges Trapezoids
SR (%) FA (%) SR (%) FA (%) SR (%) FA (%) SR (%) FA (%) SR (%) FA (%)
✗ ✗ ✗ 0.00(±0.0)(\pm 0.0) 10.62(±1.4)(\pm 1.4) 18.78(±1.7)(\pm 1.7) 77.27(±1.3)(\pm 1.3) 67.28(±1.5)(\pm 1.5) 86.71(±0.8)(\pm 0.8) 89.34(±1.0)(\pm 1.0) 92.99(±0.9)(\pm 0.9) 85.22(±1.1)(\pm 1.1) 80.43(±1.2)(\pm 1.2)
✓ ✗ ✗ 3.32(±0.6)(\pm 0.6) 60.72(±1.6)(\pm 1.6) 17.18(±1.5)(\pm 1.5) 83.63(±1.1)(\pm 1.1) 44.94(±1.8)(\pm 1.8) 88.91(±0.9)(\pm 0.9) 91.78(±0.7)(\pm 0.7) 92.43(±0.8)(\pm 0.8) 41.01(±1.9)(\pm 1.9) 83.68(±1.1)(\pm 1.1)
✓ ✓ ✗ 63.30(±1.8)(\pm 1.8) 90.78(±1.0)(\pm 1.0) 90.08(±0.5)(\pm 0.5) 94.76(±0.8)(\pm 0.8) 93.58(±0.6)(\pm 0.6) 92.70(±1.1)(\pm 1.1) 93.45(±0.9)(\pm 0.9) 91.13(±1.0)(\pm 1.0) 91.82(±0.4)(\pm 0.4) 87.97(±1.1)(\pm 1.1)
✓ ✓ ✓ 99.72(±0.3)(\pm 0.3) 93.87(±0.8)(\pm 0.8) 99.60(±0.4)(\pm 0.4) 93.92(±0.7)(\pm 0.7) 98.61(±0.5)(\pm 0.5) 92.79(±1.0)(\pm 1.0) 99.94(±0.2)(\pm 0.2) 92.94(±0.8)(\pm 0.8) 99.96(±0.2)(\pm 0.2) 87.67(±1.0)(\pm 1.0)

Complete original Figure 8: Comparison with and without the symmetry loss. Symmetry regularization promotes coordinated left–right leg alternation and improves traversal stability on challenging sparse terrains.
Figure 8: Comparison with and without the symmetry loss. Symmetry regularization promotes coordinated left–right leg alternation and improves traversal stability on challenging sparse terrains.

A.2 Analysis of Symmetry Regularization

Figure 8 contrasts a failed, poorly coordinated traversal without symmetry regularization against a successful example with it. It illustrates the motivation for avoiding synchronized hops on small supports.

The paired illustration is qualitative. The quantitative evidence comes from the box-terrain comparison and measured contact patterns, rather than an independent count of every gait failure mode.

A.3 Full Real-World Results

Figure 9 retains all fifteen original snapshots across boxes, trapezoids, stakes, beams and wedges. The gallery shows selected support transitions and body adjustments.

It should be read alongside the ten-trial hardware protocol, rather than as an additional estimate of success probability.

Complete original Figure 9: Additional real-world traversal sequences across all five terrain types. From top to bottom, the rows show boxes, trapezoids, stakes, beams, and wedges.
Figure 9: Additional real-world traversal sequences across all five terrain types. From top to bottom, the rows show boxes, trapezoids, stakes, beams, and wedges.

Appendix B Implementation Details of Policy Learning

Appendix B preserves the complete evaluation thresholds, observation specifications, reward formulas/weights and optimization settings.

B.1 Simulation Setup and Evaluation Protocol

The appendix restates the 2,048-agent asymmetric PPO setup and five-family 20×10 curriculum. Difficulty changes support spacing, height, size and orientation.

Detailed terrain parameter ranges and a held-out layout split are not specified in this document.

B.2 Evaluation Protocol

The configuration lists a 12-second timeout, a 2 m inward boundary buffer, four-second stagnation termination, 1 N torso-contact threshold, 1 rad tilt threshold and 0.9 m minimum relative root height. All original entries are retained.

The four-second stagnation value conflicts with the main text’s three seconds. Relative root height uses the environment-origin correction defined here; these are episode definitions, rather than runtime safety guarantees.

Complete original Table 5: Episode truncation and failure termination thresholds.
Table 5: Episode truncation and failure termination thresholds.
Accessible table and original citation links
Condition Threshold
Episode Truncation
Timeout 12.0​s12.0\,\mathrm{s}
Terrain boundary 2.0​m2.0\,\mathrm{m}
Failure Termination
Starting-platform stagnation 4.0​s4.0\,\mathrm{s}
Torso contact 1.0​N1.0\,\mathrm{N}
Excessive base tilt 1.0​rad1.0\,\mathrm{rad}
Insufficient root height 0.9​m0.9\,\mathrm{m}

B.3 Observation Space and Action Interface

The actor has 96 proprioceptive values per frame, stacked over eight frames for 768 values, plus four 18×32 depth tensors. Joint positions, velocities and actions each have 29 components. Noise ranges and scaling remain in the complete table.

The critic adds privileged base linear velocity and uses clean state, giving 99 values per frame. The actor does not need that privileged velocity measurement at deployment. Blur, depth normalization and delayed history are listed without complete distribution/timing specifications.

Complete original Table 6: Observation space for policy learning.
Table 6: Observation space for policy learning.
Accessible table and original citation links
Group Observation Term Base Dim. History / Shape Noise / Processing
Actor Base angular velocity 3 8×38\times 3 Uniform noise [−0.2,0.2][-0.2,0.2], scale 0.250.25
Actor Projected gravity 3 8×38\times 3 Uniform noise [−0.05,0.05][-0.05,0.05]
Actor Velocity command 3 8×38\times 3 None
Actor Joint position 29 8×298\times 29 Uniform noise [−0.01,0.01][-0.01,0.01]
Actor Joint velocity 29 8×298\times 29 Uniform noise [−0.5,0.5][-0.5,0.5], scale 0.050.05
Actor Last action 29 8×298\times 29 None
Actor Depth image 18×3218\times 32 4×18×324\times 18\times 32 Blur, depth normalization, delayed history
Critic Base linear velocity 3 8×38\times 3 None
Critic Base angular velocity 3 8×38\times 3 Scale 0.250.25
Critic Projected gravity 3 8×38\times 3 None
Critic Velocity command 3 8×38\times 3 None
Critic Joint position 29 8×298\times 29 None
Critic Joint velocity 29 8×298\times 29 Scale 0.050.05
Critic Last action 29 8×298\times 29 None
Critic Depth image 18×3218\times 32 4×18×324\times 18\times 32 Blur, depth normalization, delayed history

B.4 Reward Function

The full reward table retains every task, regularization, joint-limit/contact and motion-prior term. It includes velocity tracking, edge penalties, foot timing/sliding, action rate, energy and an AMP style reward.

The row named “Heading Error” uses commanded yaw-rate magnitude rather than an explicit measured heading discrepancy. Its original label and expression are preserved without silently changing the objective. Reward penalties should not be read as guaranteed constraint enforcement.

Equation
r=rtask+rreg+rsafe+ramp.r=r_{\mathrm{task}}+r_{\mathrm{reg}}+r_{\mathrm{safe}}+r_{\mathrm{amp}}.
Complete original Table 7: Detailed specification of reward terms, weights, and mathematical formulations.
Table 7: Detailed specification of reward terms, weights, and mathematical formulations.
Accessible table and original citation links
Reward Term Weight Mathematical Formulation
Task Reward
Don’t Wait −1.5-1.5 𝕀⁡(vx∗>0.3)​(𝕀⁡(vx<0.15)+𝕀⁡(vx<0)+𝕀⁡(vx<−0.15))\mathbb{I}(v_{x}^{*}>0.3)\,(\mathbb{I}(v_{x}<0.15)+\mathbb{I}(v_{x}<0)+\mathbb{I}(v_{x}<-0.15))
Is Alive 3.03.0 +1+1
Stand Still −0.3-0.3 (‖q−qd​e​f​a​u​l​t‖1−4.0)​𝕀​(‖v∗‖<0.15)​𝕀​(|ωz∗|<0.15)\big(\|q-q_{default}\|_{1}-4.0\big)\,\mathbb{I}(\|v^{*}\|<0.15)\,\mathbb{I}(|\omega_{z}^{*}|<0.15)
Linear Vel Tracking (vx​y)(v_{xy}) 2.02.0 exp(−∥vx​y∗−vx​y∥2/0.52)\exp\!\big(-\|v_{xy}^{*}-v_{xy}\|^{2}/0.5^{2}\big)
Angular Vel Tracking (ωz)(\omega_{z}) 0.10.1 exp(−(ωz∗−ωz)2/0.52)\exp\!\big(-(\omega_{z}^{*}-\omega_{z})^{2}/0.5^{2}\big)
Heading Error −1.0-1.0 |ωz∗||\omega_{z}^{*}|
Regularization Reward
Edge Penetration −1.0-1.0 ∑i=1|P||di|(‖vi‖+ϵ)\sum_{i=1}^{|P|}\|d_{i}\|\,(\|v_{i}\|+\epsilon)
Feet Air Time 1.51.5 min⁡(tphase,f,𝕀⁡(∑cf=1)​𝕀​(‖v∗‖>0.15))\min\!\big(t_{\mathrm{phase},f},\,\mathbb{I}(\sum c_{f}=1)\,\mathbb{I}(\|v^{*}\|>0.15)\big)
Feet Slide −0.4-0.4 ∑f‖vx​y,f‖​𝕀​(cf)\sum_{f}\|v_{xy,f}\|\,\mathbb{I}(c_{f})
Joint Deviation (Hip) −0.5-0.5 ∑jehips​(qj−qj,d​e​f​a​u​l​t)2\sum_{j}e_{\mathrm{hips}}(q_{j}-q_{j,default})^{2}
Base Ang Vel (ℓ2)(\ell_{2}) −0.5-0.5 ‖ωx​y‖2\|\omega_{xy}\|^{2}
Joint Torques (ℓ2)(\ell_{2}) −1.5×10−7-1.5{\times}10^{-7} ‖τlegs‖2\|\tau_{\mathrm{legs}}\|^{2}
Joint Acc (ℓ2)(\ell_{2}) −1.25×10−7-1.25{\times}10^{-7} ‖q¨‖2\|\ddot{q}\|^{2}
Joint Vel (ℓ2)(\ell_{2}) −1.0×10−4-1.0{\times}10^{-4} ‖q˙‖2\|\dot{q}\|^{2}
Action Rate (ℓ2)(\ell_{2}) −0.005-0.005 ‖at−at−1‖2\|a_{t}-a_{t-1}\|^{2}
Flat Orientation −3.0-3.0 ‖𝐠x​yproj‖2\|\mathbf{g}_{xy}^{\mathrm{proj}}\|^{2}
Pelvis Orientation −3.0-3.0 ‖𝐠x​yproj,pelvis‖2\|\mathbf{g}_{xy}^{\mathrm{proj},\,\mathrm{pelvis}}\|^{2}
Feet Orientation −0.4-0.4 ∑f‖𝐠x​y,fproj‖​𝕀​(cf)\sum_{f}\|\mathbf{g}_{xy,f}^{\mathrm{proj}}\|\,\mathbb{I}(c_{f})
Feet Height Error −0.1-0.1 ∑f∑pclip⁡(hf−ht​e​r​r,p−0.035, 0, 0.3)​𝕀​(cf)\sum_{f}\sum_{p}\mathrm{clip}(h_{f}-h_{terr,p}-0.035,\,0,\,0.3)\,\mathbb{I}(c_{f})
Feet Distance 1.01.0 exp⁡(−max⁡(0, 0.12−|yRL−yRR|)0.05)−1\exp\!\Big(-\dfrac{\max(0,\,0.12-|y_{R}^{L}-y_{R}^{R}|)}{0.05}\Big)-1
Energy Consumption −5.0×10−5-5.0{\times}10^{-5} ∑j(τj​q˙jkj)2\sum_{j}\Big(\dfrac{\tau_{j}\dot{q}_{j}}{k_{j}}\Big)^{2}
Freeze Upper Body −0.004-0.004 ‖qupper−qupperd​e​f​a​u​l​t‖1\|q_{\mathrm{upper}}-q_{\mathrm{upper}}^{default}\|_{1}
Safety Reward
Joint Pos Limits −1.5-1.5 ∑j(max⁡(0,qj−qj,m​a​x)+max⁡(0,qj,m​i​n−qj))\sum_{j}\!\big(\max(0,q_{j}-q_{j,max})+\max(0,q_{j,min}-q_{j})\big)
Joint Vel Limits −1.5-1.5 ∑jmax⁡(0,|q˙j|−0.9​q˙j,m​a​x)\sum_{j}\max\!\big(0,|\dot{q}_{j}|-0.9\,\dot{q}_{j,max}\big)
Torque Limits −0.01-0.01 ∑jmax⁡(0,|τj|−0.8​τj,m​a​x)2\sum_{j}\max\!\big(0,|\tau_{j}|-0.8\,\tau_{j,max}\big)^{2}
Undesired Contacts −1.0-1.0 𝕀⁡(count⁡(collisionb​o​d​y∖f​e​e​t)>0)\mathbb{I}\!\big(\mathrm{count}(\mathrm{collision}_{body\setminus feet})>0\big)
AMP Reward
AMP Style 0.250.25 max⁡[0, 1−0.25​(D⁡(St)−1)2]\max\!\big[0,\,1-0.25\,(D(S_{t})-1)^{2}\big]

B.5 Training Details

Each update collects 24 steps from 2,048 environments, or 49,152 transitions. PPO uses five epochs, four minibatches, learning rate 10⁻³ with an adaptive schedule, discount 0.99, GAE 0.95, clip 0.2 and desired KL 0.01; the maximum is 50,000 iterations.

Mirror-loss weight is 10.0, alternation weight 0.1 and alternation speed threshold 0.25. The complete table includes entropy/value weights and gradient clipping. A maximum iteration count is not a reported wall-clock training duration.

Complete original Table 8: Hyperparameters used for policy training.
Table 8: Hyperparameters used for policy training.
Accessible table and original citation links
Hyperparameter Value
Rollout length per environment 24
Maximum training iterations 50,000
Learning epochs per update 5
Mini-batches per update 4
Learning rate 1.0×10−31.0\times 10^{-3}
Learning rate schedule Adaptive
Discount factor γ\gamma 0.99
GAE parameter λ\lambda 0.95
PPO clipping parameter 0.2
Desired KL divergence 0.01
Maximum gradient norm 1.0
Value loss coefficient 1.0
Entropy coefficient 0.006
Use clipped value loss True
Symmetry loss coefficient 10.0
Alternation speed threshold 0.25
Alternation leg coefficient 0.1
Concept demonstration

Open image ↗