Literature
Display
← All papers

Shape Your Body: Value Gradients for Multi-Embodiment Robot Design

Nico Bohlinger · Jan Peters

Preprint posted · CoRL 2026 (accepted) ↗

Robotics13 min text · CondensedOriginal paper ↗
Contents

Overview

Project resources. This edition follows the complete 33-page, 13 October 2026 v2, including seventeen figures, sixteen numbered tables, eleven equations and all 62 references.

Complete original PDF block S0.F1
Figure 1: Shape Your Body. We first train an embodiment-aware policy and value function with multi-embodiment reinforcement learning, then we optimize new designs by differentiating through the value function and applying the gradients inside a soft trust region around a reference design.

At a glance

Shape Your Body trains an embodiment-aware locomotion policy and critic across randomized robots, then freezes both and differentiates the critic to optimize continuous design parameters. A penalty toward a reference design reduces exploitation of critic errors. Search improves many perturbed designs in simulation and can reuse one model across robots, but optimized hardware is neither manufactured nor tested; real deployments use the nominal robots.

1 Introduction

Robot co-design couples the body with its controller. Repeating control training for every candidate is expensive, so this work amortizes that cost through a generalist policy and value model. The frozen critic becomes a differentiable surrogate over masses, geometry, joint limits, actuation and control settings.

Experiments progress from randomized versions of one robot to a held-out robot within a morphology class, then a model trained on all fifty robots. The last setting includes the targets and is not held-out generalization. Gradients also identify groups of design changes; these should not automatically be interpreted as isolated causal importance.

2 Related Work

Early evolutionary morphology and automated robot design jointly searched bodies and behavior. Recent FEACRL, Transform2Act, BodyGen and Stackelberg PPO share experience across designs within a training run. Here a broad pretrained critic is reused for subsequent design problems.

URMA and embodiment scaling provide the multi-robot representation. Comparators include black-box optimizers and gradient-based GC-PFO, the strongest search baseline. The idea is also related to policy evaluation networks and explicit policy-conditioned value functions, which differentiate critics with respect to representations of what they evaluate.

Complete original figure S2.F2
Figure 2: Optimized robot designs. We visualize two initial designs and the corresponding optimized designs for the Go2 and the MIT Humanoid. We also show the reference design and an optimized design for the Fourier GR1 T2 humanoid. Although some assets appear stretched or visually disconnected, the underlying geometries remain connected and controllable by the policy, as discussed in Appendix M. Full optimization trajectories across initial designs for our main three robots are shown in Appendix N.

3 Value Gradients for Multi-Embodiment Robot Design

A fixed-topology robot has physical parameters e=Φ(f), with normalized design f in [−1,1]ᵈ. Equation 1 defines discounted policy return; search aims to improve a candidate under a frozen embodiment-conditioned controller. It can alter geometry, dynamics, actuator limits and PD settings, but cannot add joints or change topology.

The differentiable map applies bounded transformations and consistency rules. These constrain the simulated model without certifying fabrication feasibility, real actuator availability or collision-free operation. Design-specific reward penalties are possible but are not added here.

Equation 1
J(θ,e)=𝔼τ∼πθ(⋅∣⋅,e)[∑t=0Tγtre(st,at)],J(\theta,e)=\mathbb{E}_{\tau\sim\pi_{\theta}(\cdot\mid\cdot,e)}\left[\sum_{t=0}^{T}\gamma^{t}r_{e}(s_{t},a_{t})\right], (1)

3.1 Training a Multi-Embodiment Policy and Critic

At episode reset, sample a design from a curriculum-scaled box and hold it fixed for the rollout. PPO trains a shared policy. URMA encodes per-joint observations/descriptions, aggregates variable-length joint/foot information and produces per-joint actions or a scalar value.

The direct-design critic feeds physical descriptions into attention values as well as keys, increasing design sensitivity. Three value heads share encoders and their mean drives search. This ensemble is not calibrated uncertainty: later tests show that heads can agree on harmful extrapolations.

Equations 2–3
maxθ⁡𝔼f∼𝒰⁡([−1,1]ddesign)​[J⁡(θ,Φ⁡(f))].\max_{\theta}\;\mathbb{E}_{f\sim\mathcal{U}([-1,1]^{d_{\mathrm{design}}})}\left[J(\theta,\Phi(f))\right]. (2)
z¯joints=∑j∈𝒥αj​(dj)⊙gψ​(oj),αj​(dj)=exp⁡(gϕ​(dj)/η)∑Ldexp⁡(gϕ​(dj)/η),\bar{z}_{\mathrm{joints}}=\sum_{j\in\mathcal{J}}\alpha_{j}(d_{j})\odot g_{\psi}(o_{j}),\qquad\alpha_{j}(d_{j})=\frac{\exp\!\left(g_{\phi}(d_{j})/\eta\right)}{\sum_{L_{d}}\exp\!\left(g_{\phi}(d_{j})/\eta\right)}, (3)

3.2 Value-Gradient Design Search

Collect a fixed state bank under the trained policy, then maximize mean predicted value minus λ times average squared distance from a reference design. Equations 4–7 retain the ensemble, constrained formulation, soft penalty and clipped Adam update. Default search uses fifty iterations, learning rate 0.05, λ=100 and maximum normalized coordinate step 0.1.

These are derivatives of a learned surrogate, not exact gradients of rollout return. The fixed bank does not follow every candidate’s occupancy distribution. The soft penalty discourages extrapolation without imposing a hard radius; final clipping enforces the design box. Ablations support the recipe without establishing convergence or performance guarantees.

Equations 4–7
V¯​(s,Φ⁡(f))=1K​∑k=1KVk​(s,Φ⁡(f)).\bar{V}(s,\Phi(f))=\frac{1}{K}\sum_{k=1}^{K}V_{k}(s,\Phi(f)). (4)
maxf∈ℱ⁡𝔼s∼𝒰⁡(𝒮)​[V¯​(s,Φ⁡(f))]s.t.‖f−fref‖22ddesign≤ϵ.\max_{f\in\mathcal{F}}\;\mathbb{E}_{s\sim\mathcal{U}(\mathcal{S})}\left[\bar{V}(s,\Phi(f))\right]\quad\text{s.t.}\quad\frac{\left\|f-f_{\mathrm{ref}}\right\|_{2}^{2}}{d_{\mathrm{design}}}\leq\epsilon. (5)
J^λ​(f)=𝔼s∼𝒰⁡(𝒮)​[V¯​(s,Φ⁡(f))]−λ​‖f−fref‖22ddesign.\hat{J}_{\lambda}(f)=\mathbb{E}_{s\sim\mathcal{U}(\mathcal{S})}\left[\bar{V}(s,\Phi(f))\right]-\lambda\frac{\left\|f-f_{\mathrm{ref}}\right\|_{2}^{2}}{d_{\mathrm{design}}}. (6)
fn+1=clip[−1,1]​(fn+clip[−δmax,δmax]​[ΔAdam​(gn)]).f_{n+1}=\mathrm{clip}_{[-1,1]}\left(f_{n}+\mathrm{clip}_{[-\delta_{\max},\delta_{\max}]}\left[\Delta_{\mathrm{Adam}}(g_{n})\right]\right). (7)

4 Experiments

The task tracks random planar-linear and yaw-velocity commands in RL-X with MuJoCo/MJX. Training spans fifteen quadrupeds, 31 bipeds/humanoids and four hexapods. Main results report return improvement from perturbed starts; Appendix I separately measures planar tracking-error reduction, which need not improve whenever the multi-term return does.

4.1 Single-Robot Design

Separate models train on randomized Go2, MIT Humanoid and Golem designs with 358, 514 and 688 parameters. Ten starts per robot are searched with the same critic objective. VGDS approximately matches GC-PFO on Go2/MIT and improves Golem most. At λ=100, mean return gains are 20.60, 28.84 and 16.85.

The nominal anchor accounts for much of this recovery: following only the reference penalty gives 19.60, 27.97 and 16.35. The gain over random designs is therefore not all critic guidance. Adapted RL co-design baselines retrain for each start; reuse adds one to two minutes per design after seven to nine hours of single-robot training. The fifty-robot run costs about 2.5 days, excluded from that small marginal search cost.

Complete original PDF block S4.F3
Figure 3: Single-robot design. Mean return improvement Δ​R\Delta R over the initial perturbed design finitf_{\mathrm{init}} for 1010 starts per robot. The nominal URDF design freff_{\mathrm{ref}} is shown as a reference, but is only used as the anchor for the trust region.
Complete original PDF block S4.F4
Figure 4: Comparison to RL-based co-design. The first three plots show the return over environment steps for Schaff2019 [24], FEACRL, Transform2Act, BodyGen, and Stackelberg PPO. The dashed line shows the final performance of VGDS after training a policy and critic on the full design space, and then designing from the same finitf_{\mathrm{init}} as the RL baselines started from. The fourth plot shows the cumulative time needed to co-design increasing numbers of initial designs.

4.2 Generalization Across Training Distributions

Class-specific training excludes the target; all-fifty training includes it. VGDS return gains with all fifty are 25.62 for Go2, 20.47 for MIT and 5.23 for Golem, versus held-out class gains 24.10, 13.75 and 13.65. GC-PFO slightly exceeds VGDS on held-out Go2/MIT (24.42/13.82), so there is no strict win in every cell.

Broader training benefits common classes but not Golem. The balancing study changes diversity and per-robot exposure too, so it does not isolate class count. Policies also have different initial/nominal returns: larger improvement across training sets need not mean higher absolute final performance.

Complete original PDF block S4.F5
Figure 5: Effect of training sets. Target robot is either held out from a morphology class set (open circles) or included in the full 50-robot training set (filled circles). We omit error bars for readability and report them in Appendix J.

4.3 Design Analysis with Value Gradients

MIT changes emphasize nominal joint poses, gains and foot size: gains alone raise return 5.8→12.5, versus 31.1 for the full design. Go2 changes leg axes, foot geometry and actuator limits. Golem lowers action scale/P gain and raises D gain, although copying its gain changes alone hurts performance.

These coupled interventions support useful analysis, not a unique causal ranking. Some humanoids become shorter and wider; the proposed stability/training-distribution explanation is not separately established.

Complete original PDF block S4.F6
Figure 6: Grouped design changes for the MIT Humanoid. Signed Root Mean Square (RMS) changes of the optimized designs by VGDS relative to the nominal reference, grouped by body part and parameter type. This can give an overview over which parameters are most influential for the design optimization. The heatmaps for the Unitree Go2 and Golem are shown in Appendix M.

5 Limitations

Topology is fixed, and results depend on critic coverage, a useful reference and a controller capable of the candidate body. Comparatively simple simulations do not establish manufacturability or hardware transfer of optimized designs. Physical demonstrations validate only nominal robots. Materials, electronics, actuation and fabrication constraints remain open.

6 Conclusion

Embodiment-aware critics make high-dimensional design search inexpensive after training and help analyze combined body/control changes. The evidence is simulated improvement under frozen controllers, with strong dependence on the reference and surrogate. Proposed extensions include topology search, richer physical constraints/objectives and latent-space trust regions.

Acknowledgments

Funding is acknowledged from Poland’s National Science Centre Weave programme, UMO-2021/43/I/ST6/02711, and the German Research Foundation under PE 2315/17-1.

Appendix

Appendix A Experimental Setup Details

A.1 Environment

The JAX/MJX environment simulates at 200 Hz and controls at 50 Hz for at most twenty seconds or 1,000 actions. Early termination starts at 80% of nominal root height and relaxes toward zero with the curriculum. Robot-scale adjustments share the remaining task setup.

Actions and PD controller.

Equation 8 adds a robot-scaled action in [−10,10] to each nominal joint position; a PD controller tracks the target. Gains and action scale are searchable, so optimized embodiments include controller settings.

Equation 8
qjtarget=qjnom+σrobot​aj,q^{\mathrm{target}}_{j}=q^{\mathrm{nom}}_{j}+\sigma_{\mathrm{robot}}a_{j}, (8)
Complete original figure A1.F7
Figure 7: Overview of all 50 robots used in the multi-embodiment RL training [45].

Observations and descriptions.

Joint observations include position offset, velocity, previous action and nominal-pose indicators. Descriptions encode axes, limits, gains, masses, inertias, geometry and global properties. Foot contacts/timing, linear velocity, terrain height and curriculum state are privileged critic inputs; the policy has a more restricted observation set.

A.2 Reward function

Equations 9–10 and Table 1 retain all 23 terms: planar/yaw tracking, survival and motion/effort/contact/joint penalties. Pre-reward is clipped below at zero before adding an unclipped survival term; coefficients scale with the control timestep. Return gains can trade tracking against other objectives.

Equations 9–10
r~t=w1​T1+w2​T2+ct​(w3​T3+∑i=523wi​Ti),\tilde{r}_{t}=w_{1}T_{1}+w_{2}T_{2}+c_{t}\left(w_{3}T_{3}+\sum_{i=5}^{23}w_{i}T_{i}\right), (9)
rt=max⁡(r~t,0)+ct​w4​T4.r_{t}=\max(\tilde{r}_{t},0)+c_{t}w_{4}T_{4}. (10)
Complete original PDF block A1.T1
Table 1: Reward terms for locomotion training. Joint-based and foot-based terms are averaged over the corresponding actuated joints or feet to keep the reward scales consistent across different morphologies.
Original selectable table
Term Equation Coef.
T1 xx-yy velocity tracking exp(−∥vx​y−cx​y∥22/0.25)\exp(-\|v_{xy}-c_{xy}\|_{2}^{2}/0.25) 2.02.0
T2 yaw\mathrm{yaw} velocity tracking exp(−(ωz−cω)2/0.25)\exp(-(\omega_{z}-c_{\omega})^{2}/0.25) 1.01.0
T3 Alive clipped 11 0.050.05
T4 Alive unclipped 11 0.050.05
T5 Z velocity penalty −|vz|2-|v_{z}|^{2} 2.02.0
T6 IMU acceleration penalty −‖Δ​vimu/Δ​t‖22-\|\Delta v_{\mathrm{imu}}/\Delta t\|_{2}^{2} 10−410^{-4}
T7 Roll-pitch velocity penalty −‖ωx​y‖22-\|\omega_{xy}\|_{2}^{2} 0.050.05
T8 Roll-pitch position penalty −‖θx​y‖22-\|\theta_{xy}\|_{2}^{2} 10.010.0
T9 Joint nominal deviation penalty −‖qu−qunom‖2¯-\overline{\|q_{u}-q^{\mathrm{nom}}_{u}\|^{2}} 20.020.0
T10 Joint position limit penalty −[ℓu−qu]++[qu−uu]+¯-\overline{[\ell_{u}-q_{u}]_{+}+[q_{u}-u_{u}]_{+}} 40.040.0
T11 Joint velocity limit penalty −[|q˙u|−0.9​q˙umax]+¯-\overline{[|\dot{q}_{u}|-0.9\dot{q}^{\max}_{u}]_{+}} 5.05.0
T12 Joint velocity penalty −‖q˙u‖2¯-\overline{\|\dot{q}_{u}\|^{2}} 4×10−44\times 10^{-4}
T13 Joint acceleration penalty −‖Δ​q˙u/Δ​t‖2¯-\overline{\|\Delta\dot{q}_{u}/\Delta t\|^{2}} 5×10−65\times 10^{-6}
T14 Joint torque penalty −‖τu‖2¯-\overline{\|\tau_{u}\|^{2}} 4×10−44\times 10^{-4}
T15 Positive power penalty −[τu​q˙u]+¯-\overline{[\tau_{u}\dot{q}_{u}]_{+}} 4×10−44\times 10^{-4}
T16 Action rate penalty −‖at−at−1‖2¯-\overline{\|a_{t}-a_{t-1}\|^{2}} 3.03.0
T17 Action smoothness penalty −‖at−2​at−1+at−2‖2¯-\overline{\|a_{t}-2a_{t-1}+a_{t-2}\|^{2}} 0.10.1
T18 Self-collision penalty −ncol-n_{\mathrm{col}} 2.02.0
T19 Base-height penalty −|h−hnom|2-|h-h^{\mathrm{nom}}|^{2} 30.030.0
T20 Foot air-time penalty 𝟙cf​min⁡(tfair−0.4​srobot,0)¯\overline{\mathbbm{1}_{c_{f}}\min(t_{f}^{\mathrm{air}}-0.4s_{\mathrm{robot}},0)} 3.03.0
T21 Symmetric air penalty −𝟙¬cfl​𝟙¬cfr¯-\overline{\mathbbm{1}_{\neg c_{f_{l}}}\mathbbm{1}_{\neg c_{f_{r}}}} 1.01.0
T22 Foot slip penalty −𝟙cf​‖vx​yf‖2¯-\overline{\mathbbm{1}_{c_{f}}\|v^{f}_{xy}\|^{2}} 0.10.1
T23 Foot z-velocity penalty −‖min⁡(vzf,0)‖2¯-\overline{\|\min(v^{f}_{z},0)\|^{2}} 0.20.2

A.3 Curriculum and design sampling

The box expands from the nominal design to the full normalized range using the performance-based embodiment curriculum. Evaluation uses the full box. Training also perturbs observations and initial states, applies pushes and mixes action delays up to one control step.

A.4 Robots and design dimensions

Figure 7 and the complete two-page Table 2 retain all fifty robots, gains, action scales, actuator counts and dimensions. STAR1 has the largest listed space, 1,177 parameters. Every robot keeps its kinematic topology.

Complete original PDF block A1.T2-1Complete original PDF block A1.T2-2
Table 2: Set of robots used for multi-embodiment training and design.
Original selectable table
Robot p-gain d-gain σrobot\sigma_{\mathrm{robot}} nun_{u} ddesignd_{\mathrm{design}}
Quadrupeds
Unitree A1 20.020.0 0.50.5 0.250.25 1212 358358
Unitree Go1 20.020.0 0.50.5 0.250.25 1212 358358
Unitree Go2 20.020.0 0.50.5 0.300.30 1212 358358
Unitree B2 80.080.0 2.02.0 0.500.50 1212 358358
ANYmal B 80.080.0 2.02.0 0.500.50 1212 358358
ANYmal C 80.080.0 2.02.0 0.500.50 1212 358358
Barkour v0 20.020.0 0.50.5 0.250.25 1212 358358
Barkour vB 30.030.0 0.50.5 0.250.25 1212 358358
Silver Badger 20.020.0 0.50.5 0.250.25 1313 385385
Honey Badger 20.020.0 0.50.5 0.250.25 1212 358358
Bittle 25.025.0 0.50.5 0.600.60 88 250250
Aibo 20.020.0 0.50.5 0.250.25 1717 493493
Solo 12 20.020.0 0.50.5 0.250.25 1212 358358
Spot 80.080.0 2.02.0 0.500.50 1212 358358
Spot Mini 20.020.0 0.50.5 0.250.25 1212 358358
Bipeds and humanoids
Bolt 15.015.0 0.30.3 0.200.20 66 190190
Open Duck Mini 6.56.5 0.50.5 0.500.50 1616 460460
Mini Pi 20.020.0 0.50.5 0.300.30 1212 352352
QMini 15.015.0 1.01.0 0.400.40 1010 298298
Tron 1 40.040.0 1.01.0 0.500.50 66 190190
Unitree H1 60.060.0 2.02.0 0.750.75 1919 541541
Unitree G1 45.045.0 1.01.0 0.500.50 2323 985985
Talos 80.080.0 2.02.0 0.750.75 3030 10201020
ROBOTIS OP3 21.021.0 0.50.5 0.600.60 2020 568568
NAO v5 30.030.0 1.01.0 0.700.70 2222 11021102
ADAM Lite 80.080.0 2.02.0 0.750.75 2525 703703
AgiBot X1 45.045.0 1.01.0 0.700.70 2929 811811
Apptronik Apollo 100.0100.0 2.02.0 0.750.75 3030 890890
Atlas 80.080.0 2.02.0 0.750.75 2727 783783
Berkeley Humanoid 20.020.0 0.60.6 0.500.50 1212 352352
Berkeley Humanoid Lite 20.020.0 0.50.5 0.300.30 2222 661661
Booster T1 25.025.0 0.60.6 0.500.50 2323 649649
ELF2 35.035.0 0.70.7 0.500.50 2525 703703
EngineAI SA01 55.055.0 1.51.5 0.500.50 1212 352352
Fourier GR1 T2 80.080.0 2.02.0 0.750.75 3434 946946
Fourier GR2 v3 80.080.0 2.02.0 0.750.75 2929 811811
Fourier N1 70.070.0 2.02.0 0.700.70 2323 649649
K-Bot v1 40.040.0 2.02.0 0.500.50 2020 594594
K-Bot v2 50.050.0 1.01.0 0.500.50 2020 594594
MIT Humanoid 30.030.0 0.50.5 0.500.50 1818 514514
Northwestern Humanoid 20.020.0 0.60.6 0.500.50 1010 298298
Poppy Humanoid 20.020.0 0.50.5 0.300.30 2525 703703
Sigmaban 21.021.0 0.50.5 0.500.50 2020 568568
STAR1 80.080.0 2.02.0 0.750.75 3131 11771177
Valkyrie 100.0100.0 2.02.0 0.750.75 2626 730730
Z-Bot 10.010.0 0.50.5 0.400.40 2020 594594
Hexapods
Crab 20.020.0 0.50.5 0.500.50 1818 526526
Custom Hexapod 30.030.0 0.50.5 0.600.60 1818 526526
Tom Hexapod 25.025.0 0.50.5 0.500.50 1818 526526
Golem 25.025.0 0.50.5 0.500.50 2424 688688

A.5 PPO Training

PPO uses 64-step rollouts, 16,384-sample minibatches, ten epochs, discount 0.99, GAE λ=0.7 and clip range 0.1. Single-robot training uses 4,096 environments/one billion steps; class training 7,680/3.7 billion; all-fifty training 15,040/3.7 billion. Hardware varies; the largest run takes about 2.5 days on one RTX 6000 Ada.

Complete original PDF block A1.T3
Table 3: PPO hyperparameters. These are the default training hyperparameters. Large multi-robot runs adjust the total number of environment steps and number of parallel environments.
Original selectable table
Hyperparameter Value
Rollout length 6464
Minibatch size 16,38416{,}384
Epochs 1010
Learning rate schedule linear, 1.5×10−3→3×10−41.5{\times}10^{-3}\rightarrow 3{\times}10^{-4}
Discount factor γ\gamma 0.990.99
GAE λ\lambda 0.70.7
Clip range 0.10.1
Entropy coefficient 0.0050.005
Max gradient norm 3.03.0
KL threshold 0.50.5
Initial action standard deviation 1.01.0
Policy mean clipping [−10,10][-10,10]
Policy std. clipping [0.01,2.0][0.01,2.0]
Parallel environments 4,0964{,}096 or 7,6807{,}680 or 15,04015{,}040
Total environment steps 1.0×1091.0{\times}10^{9} or 3.7×1093.7{\times}10^{9}

A.6 Design space parameters

Table 4 gives every group, range and transform. Multiplicative groups use x=x_nom(1+κfᵢ); additive groups use x=x_nom+κfᵢ; rotations use fixed axes. For body/joint/actuator/foot counts n_b,n_j,n_u,n_f, dimension is 9+13n_b+11n_j+3n_u+3n_f.

Body-axis scaling also changes positions and derived mass/inertia/control properties; separate groups further vary masses and inertias. Damping, armature, stiffness and friction are nonnegative; nominal joint positions obey limits. Closed-loop robots disable body size, position and orientation changes. These are simulation constraints, not a complete engineering-feasibility certificate.

Complete original PDF block A1.T4
Table 4: Parameter groups of the design space. We show the dimension, transform, and range κ\kappa of each group for a robot with nbn_{b} bodies, njn_{j} joints, nun_{u} actuated joints, and nfn_{f} feet.
Original selectable table
Parameter group Dimension Transform Range
Robot
Body size per axis 33 mult. ±80%\pm 80\%
IMU position 33 add. ±0.1\pm 0.1 m
Bodies
Coupled mass and inertia nbn_{b} mult. ±25%\pm 25\%
Decoupled mass nbn_{b} mult. ±25%\pm 25\%
Decoupled inertia 3​nb3n_{b} mult. ±25%\pm 25\%
Center of mass 3​nb3n_{b} add. ±0.02\pm 0.02 m
Inertia orientation nbn_{b} rot. ±0.2\pm 0.2 rad
Body position 3​nb3n_{b} add. ±0.02\pm 0.02 m
Body orientation nbn_{b} rot. ±0.02\pm 0.02 rad
Feet
Foot size 3​nf3n_{f} mult. ±75%\pm 75\%
Joints
Joint axis njn_{j} rot. ±0.1\pm 0.1 rad
Joint range limits 2​nj2n_{j} add. ±0.4\pm 0.4 rad
Damping njn_{j} mult. ±100%\pm 100\%
Damping njn_{j} add. ±0.1\pm 0.1 N m s/rad
Armature njn_{j} mult. ±100%\pm 100\%
Armature njn_{j} add. ±0.001\pm 0.001 kg m2
Stiffness njn_{j} mult. ±10%\pm 10\%
Stiffness njn_{j} add. ±0.1\pm 0.1 N m/rad
Friction loss njn_{j} mult. ±100%\pm 100\%
Friction loss njn_{j} add. ±10−5\pm 10^{-5} N m
Actuators
Torque limit nun_{u} mult. ±75%\pm 75\%
Velocity limit nun_{u} mult. ±75%\pm 75\%
Nominal joint position nun_{u} add. ±0.1\pm 0.1 rad
Controller
P gain 11 mult. ±50%\pm 50\%
D gain 11 mult. ±50%\pm 50\%
Action scaling factor 11 mult. ±50%\pm 50\%

Appendix B Direct-Design URMA Critic

Descriptions enter the critic’s value encoders directly. A 32-dimensional joint-value latent with three heads gives +17.76 improvement in the early Go2 ablation, versus +13.57 for the original three-head critic at dimension four. Direct conditioning with four dimensions is worse (+6.41), and 64 dimensions falls to +14.35. Architecture and capacity both matter.

Figure 8 and Tables 5–6 retain the details: about 5.38 million critic and 0.95 million policy parameters. Shared heads use weight normalization. This small ablation does not establish universal superiority across robot distributions.

Complete original PDF block A2.F8
Figure 8: Overview of the direct-design URMA critic architecture. We simplify the visualization: The foot encoder corresponds to the joint encoder, but with foot observations and foot description vectors as input, and the value heads all use the same structure. The linear layers in the value heads are all wrapped in WeightNorm (WN) layers [61], which improves the conditioning of the critic optimization [62].
Complete original PDF block A2.T5
Table 5: Critic architecture ablation for gradient-based design on an early-stage Unitree Go2 design task, with results averaged over 10 initial designs.
Original selectable table
Critic Heads Latent dim. Δ​R\Delta R
Original URMA critic 1 4 +10.39
Original URMA critic 3 4 +13.57
Original URMA critic 3 8 +12.59
Original URMA critic 3 16 +10.22
Direct-design critic 3 4 +6.41
Direct-design critic 3 8 +9.12
Direct-design critic 3 16 +10.98
Direct-design critic 3 32 +17.76
Direct-design critic 3 64 +14.35
Complete original PDF block A2.T6
Table 6: Layer widths and parameter counts of policy and critic.
Original selectable table
Block Critic Policy
Joint encoder 64, 64 256, 256
Joint value latent 32 4
Foot encoder 32, 32 —
Foot value latent 32 —
Head MLP 512, 256, 128, 128, 128, 1 512, 256, 256, 256, 256
# heads 3 1
Total parameters 5,378,3125{,}378{,}312 945,430945{,}430

Appendix C Surrogate Optimizer Hyperparameters

All methods optimize the same λ=100 frozen-critic objective with 64-state minibatches. Final designs receive two hundred evaluation rollouts. Hyperparameters were tuned on early versions of the three perturbed-robot tasks. Budgets are roughly comparable, not exactly equal in gradient, critic or simulator cost.

Value-Gradient Design Search.

VGDS applies fifty Adam ascent steps at learning rate 0.05, clips each coordinate change to 0.1 and clips the design to [−1,1]. It returns the final iterate, not a certified global optimum.

Random search.

Random search scores 3,200 uniform designs and selects the highest surrogate objective, without rollout-based optimization feedback.

Bayesian optimization.

BO uses BoTorch SingleTaskGP, a Matérn kernel and qLogNoisyExpectedImprovement. Ten Sobol points include the start, followed by fifty iterations. Acquisition optimization uses eight restarts, 256 raw samples and one candidate per batch.

Covariance matrix adaptation evolution strategy.

CMA-ES uses diagonal covariance for fifty generations, initial standard deviation 0.5 and population 4+floor(3 ln d), reducing the high-dimensional covariance cost.

Trust-region Bayesian optimization.

The baseline labeled TuRBO is adaptive trust-region random search here: fifty rounds of 64 candidates, radius initially 0.1 with range 0.005–0.5, shrinking by 0.7 or expanding by 1.3. This is not a full reproduction of the original Bayesian-optimization method.

Particle swarm optimization.

Global-best PSO uses 32 particles for fifty iterations, with inertia 0.9, cognitive coefficient 0.5 and social coefficient 0.3.

Cross-entropy method.

CEM samples 32 particles for fifty iterations, refits a diagonal Gaussian to the best 20%, and uses initial/floor standard deviations 0.5/0.02.

Differential evolution.

DE uses 32 particles for fifty iterations, differential weight 0.5 and binomial-crossover probability 0.9. Trials replace particles only when the surrogate improves.

Augmented random search.

ARS uses sixteen antithetic finite-difference directions for fifty iterations, perturbation standard deviation 0.1 and learning rate 0.05, retaining all directions.

Gradient-covariance particle filter optimization.

GC-PFO uses two regions of 32 particles for fifty iterations. Objective-weighted resampling and gradient-covariance noise use step 0.2, temperature 1.0 and ε=10⁻⁸. It uses gradients and is not a derivative-free comparator.

Appendix D Trust-Region and Reference-Design Ablations

Without regularization, Go2 mean improvement is −18.93; λ=100 gives +20.60. MIT/Golem also benefit, although larger penalties give their best tested values. Penalty-only updates recover nearly as much as the full method, emphasizing the strong nominal anchor.

Starting at the nominal body improves most Table 8 settings, but held-out MIT falls 14.52→14.34 and all-robot Golem 7.60→7.58. With degraded anchors, λ=20 gives Go2/MIT/Golem returns 37.16/18.49/7.00; λ=100 worsens Golem from 2.33 to 0.06. Robustness to poor anchors is therefore conditional. Exact λ-invariance under Adam, asserted for penalty-only updates, also idealizes finite numerical ε; the reported ablation is the practical evidence.

Complete original PDF block A4.T7
Table 7: Trust-region sensitivity. Mean return improvement Δ​R\Delta R of VGDS on the single-robot design setting as a function of the trust-region penalty coefficient λ\lambda over 1010 initial designs.
Original selectable table
Robot λ=0\lambda=0 λ=5\lambda=5 λ=10\lambda=10 λ=20\lambda=20 λ=50\lambda=50 λ=100\lambda=100 λ=150\lambda=150 λ=200\lambda=200 λ=300\lambda=300 TR-only freff_{\mathrm{ref}}
Go2 -18.93 14.51 14.30 14.18 13.93 20.60 14.18 14.12 13.87 19.60 19.81
MIT 2.21 29.08 29.77 30.39 30.95 28.84 31.15 31.20 30.97 27.97 28.09
Golem 0.50 6.03 9.94 9.73 14.08 16.85 15.63 16.46 17.27 16.35 16.60
Complete original PDF block A4.T8
Table 8: Starting at the reference design. Return of the reference design freff_{\mathrm{ref}} and of the design found by VGDS when the search starts directly at freff_{\mathrm{ref}}.
Original selectable table
Go2 MIT Golem
Training set freff_{\mathrm{ref}} VGDS freff_{\mathrm{ref}} VGDS freff_{\mathrm{ref}} VGDS
Single robot 41.0941.09 42.04\mathbf{42.04} 30.3430.34 30.88\mathbf{30.88} 19.6119.61 20.21\mathbf{20.21}
Morphology class (held out) 37.8337.83 39.00\mathbf{39.00} 14.52\mathbf{14.52} 14.3414.34 1.271.27 24.65\mathbf{24.65}
All 50 robots 37.3337.33 39.18\mathbf{39.18} 17.4017.40 20.49\mathbf{20.49} 7.60\mathbf{7.60} 7.587.58
Complete original PDF block A4.T9
Table 9: Degraded reference designs. Return of random reference designs fref′=finitf^{\prime}_{\mathrm{ref}}=f_{\mathrm{init}} and of the designs found by VGDS when the trust region is anchored at fref′f^{\prime}_{\mathrm{ref}} instead of the nominal design. We show the mean ±\pm standard deviation over 3 initial designs in the single-robot design setting.
Original selectable table
Go2 MIT Golem
fref′f^{\prime}_{\mathrm{ref}} 27.59±4.2727.59\pm 4.27 0.19±0.260.19\pm 0.26 2.33±3.222.33\pm 3.22
VGDS with λ=20\lambda=20 37.16±1.69\mathbf{37.16\pm 1.69} 18.49±11.41\mathbf{18.49\pm 11.41} 7.00±4.95\mathbf{7.00\pm 4.95}
VGDS with λ=100\lambda=100 34.31±3.1734.31\pm 3.17 16.90±10.9816.90\pm 10.98 0.06±0.080.06\pm 0.08

Appendix E Critic Reliability Analysis

The critic can be exploited even when random-design averages look sensible. Scaling the found direction eventually destroys rollout return while value stays optimistic: MIT at α=5 returns 4.4 versus 30.7 at α=1. In unregularized Go2 search, return falls 27.6→8.7 while predicted value rises 2.4→6.2.

Head disagreement sometimes warns of failure, but Appendix O also reports confident catastrophic extrapolation. Three random samples per robot give limited evidence of average trends, not calibration guarantees. Discounted value and reported rollout return have different scales. Scaling α above one can leave the normal box; it is a stress test, not the bounded main search.

Complete original PDF block A5.T10
Table 10: Critic reliability along the found design direction. Rollout return RR, mean value prediction V¯\bar{V}, and critic head disagreement σV\sigma_{V} for designs f=α⋅f⋆f=\alpha\cdot f^{\star} in the single-robot design setting.
Original selectable table
Robot α=0\alpha=0 α=1\alpha=1 α=2\alpha=2 α=3\alpha=3 α=5\alpha=5 α=8\alpha=8
Go2 RR 41.041.0 42.142.1 42.442.4 42.442.4 38.638.6 0.70.7
V¯\bar{V} 5.635.63 5.805.80 6.026.02 5.985.98 4.524.52 1.601.60
σV\sigma_{V} 0.040.04 0.010.01 0.120.12 0.120.12 0.610.61 0.680.68
MIT RR 30.430.4 30.730.7 31.431.4 30.630.6 4.44.4 0.00.0
V¯\bar{V} 5.325.32 5.415.41 5.465.46 5.485.48 5.035.03 0.750.75
σV\sigma_{V} 0.000.00 0.000.00 0.010.01 0.020.02 0.160.16 0.410.41
Golem RR 19.719.7 19.819.8 21.221.2 9.69.6 5.05.0 0.00.0
V¯\bar{V} 4.964.96 5.085.08 5.205.20 5.345.34 2.202.20 0.740.74
σV\sigma_{V} 0.020.02 0.010.01 0.080.08 0.230.23 1.261.26 0.460.46
Complete original PDF block A5.T11
Table 11: Critic exploitation over search iterations. Mean value prediction V¯\bar{V} from the start to the end of the search and rollout return RR of the intermediate designs ftf_{t}. We show the mean over 3 initial designs in the single-robot design setting.
Original selectable table
V¯\bar{V} (t=0→50t{=}0\to 50) RR at iteration tt
00 1010 2020 3030 4040 5050
Go2, λ=100\lambda{=}100 2.4→5.82.4\to 5.8 27.627.6 40.340.3 41.641.6 41.441.4 41.541.5 41.841.8
Go2, λ=0\lambda{=}0 2.4→6.22.4\to 6.2 27.627.6 25.525.5 14.114.1 12.312.3 11.911.9 8.78.7
MIT, λ=100\lambda{=}100 2.3→5.42.3\to 5.4 0.20.2 27.827.8 29.629.6 30.830.8 31.531.5 31.431.4
MIT, λ=0\lambda{=}0 2.3→5.62.3\to 5.6 0.20.2 10.110.1 14.314.3 7.97.9 2.52.5 2.42.4
Golem, λ=100\lambda{=}100 0.6→5.20.6\to 5.2 2.32.3 11.311.3 17.717.7 17.817.8 18.318.3 20.020.0
Golem, λ=0\lambda{=}0 0.6→5.50.6\to 5.5 2.32.3 4.74.7 5.55.5 5.45.4 4.14.1 2.82.8
Complete original PDF block A5.T12
Table 12: Critic reliability at increasing distance from the reference design. Rollout return RR and mean value prediction V¯\bar{V} of random designs from the full design space, scaled by a factor of 0.250.25–1.01.0 toward the reference design. We show the mean and ±\pm standard deviation over 3 random designs per robot in the single-robot design setting.
Original selectable table
Design scaling factor
Robot 0.250.25 0.50.5 0.750.75 1.01.0
Go2 RR 40.1±1.840.1\pm 1.8 36.7±4.136.7\pm 4.1 29.5±7.929.5\pm 7.9 19.0±13.119.0\pm 13.1
V¯\bar{V} 5.60±0.085.60\pm 0.08 5.33±0.185.33\pm 0.18 4.13±0.474.13\pm 0.47 2.35±0.992.35\pm 0.99
MIT RR 27.9±3.227.9\pm 3.2 13.1±11.113.1\pm 11.1 8.2±10.88.2\pm 10.8 5.0±7.05.0\pm 7.0
V¯\bar{V} 5.32±0.015.32\pm 0.01 5.24±0.065.24\pm 0.06 4.47±0.524.47\pm 0.52 1.86±0.661.86\pm 0.66
Golem RR 18.7±2.318.7\pm 2.3 16.7±4.116.7\pm 4.1 12.5±3.412.5\pm 3.4 6.3±3.76.3\pm 3.7
V¯\bar{V} 4.76±0.054.76\pm 0.05 3.96±0.233.96\pm 0.23 2.29±0.492.29\pm 0.49 1.00±0.051.00\pm 0.05

Appendix F Policy Fine-Tuning at Optimized Designs

Fine-tuning for 52 million steps at the optimized body changes Go2 41.99→41.87, MIT 31.11→34.49 and Golem 20.34→39.10. Frozen policies can thus underestimate a design’s attainable performance. Fine-tuning initial Golem instead collapses 12.43→0.00. The absent Go2 gain is not proof of global policy optimality.

Complete original PDF block A6.T13
Table 13: Policy fine-tuning at fixed designs. Return of the frozen policy and of the policy fine-tuned for 52M steps at the fixed initial and optimized designs. We show the mean and ±\pm standard deviation over 3 seeds for the fine-tuned policies in the single-robot design setting.
Original selectable table
finitf_{\mathrm{init}} f⋆f^{\star}
Robot Frozen Fine-tuned Frozen Fine-tuned
Go2 22.9422.94 28.47±0.2428.47\pm 0.24 41.99\mathbf{41.99} 41.87±0.0841.87\pm 0.08
MIT 5.795.79 5.74±0.135.74\pm 0.13 31.1131.11 34.49±0.17\mathbf{34.49\pm 0.17}
Golem 12.4312.43 0.00±0.000.00\pm 0.00 20.3420.34 39.10±0.21\mathbf{39.10\pm 0.21}

Appendix G Class-Balanced Training Distribution

A balanced twelve-robot set includes four per class and the targets. Go2/MIT/Golem gains are 23.33/15.03/6.79, versus all-fifty 25.62/20.47/5.23. Held-out hexapod-only training reaches 13.65 for Golem but uses about 4.5 times more steps per robot. Balance, diversity, target inclusion and exposure remain entangled.

Complete original PDF block A7.T14
Table 14: Class-balanced training distribution. Return improvement Δ​R\Delta R of VGDS with critics trained on the target robot’s morphology class, on the class-balanced 12-robot set, and on all 50 robots. The morphology class critics hold the target robot out, while the other two training sets include it. We show the mean and ±\pm standard deviation over the same 10 initial designs.
Original selectable table
Robot Morphology class (held out) Balanced 12 All 50 robots
Go2 24.10±10.9024.10\pm 10.90 23.33±9.7823.33\pm 9.78 25.62±10.94\mathbf{25.62\pm 10.94}
MIT 13.75±1.0913.75\pm 1.09 15.03±0.8815.03\pm 0.88 20.47±1.30\mathbf{20.47\pm 1.30}
Golem 13.65±9.33\mathbf{13.65\pm 9.33} 6.79±3.036.79\pm 3.03 5.23±3.445.23\pm 3.44

Appendix H Sim-to-Real Transfer of the Training Pipeline

The all-fifty policy is deployed on nominal Unitree Go2, G1 and H1 for ordinary omnidirectional walking. Figure 9 documents qualitative transfer. No optimized body is manufactured/tested, and no quantitative hardware tracking comparison is supplied.

Complete original figure A8.F9
Figure 9: Real-world deployment of the multi-embodiment policy at the nominal designs. We test typical omnidirectional walking on the Unitree Go2, Unitree G1, and Unitree H1.

Appendix I Velocity Tracking Error Reduction Results

Equation 11 defines positive improvement as reduced absolute planar velocity error. VGDS and GC-PFO are similar on single-robot tasks. Training-set trends broadly follow return gains but differ in some rankings: multi-term return need not imply better tracking than the nominal body.

The Golem x-axis label in Figures 10–11 is truncated in both the published paper and its original web figure; the axis is tracking-error reduction in m/s, as defined above.

Equation 11
Δ​ex​y=ex​y​(finit)−ex​y​(f⋆),\Delta e_{xy}=e_{xy}(f_{\mathrm{init}})-e_{xy}(f^{\star}), (11)
Complete original PDF block A9.F10
Figure 10: xx-yy tracking error reduction for single-robot design. Mean reduction Δ​ex​y\Delta e_{xy} in absolute xx-yy linear velocity tracking error relative to the initial perturbed design finitf_{\mathrm{init}}, for 1010 starts per robot.
Complete original PDF block A9.F11
Figure 11: xx-yy tracking error reduction for multi-robot design. Mean reduction Δ​ex​y\Delta e_{xy} in absolute xx-yy linear velocity tracking error when the target robot is either held out from its morphology class training set or included in the full 50-robot training set. We omit error bars for readability and report them in Appendix J.

Appendix J Numeric Results for the Training Set Comparison

Tables 15–16 retain all means and standard deviations across ten starts. VGDS held-out/all-fifty tracking reductions are 0.19/0.18 m/s for Go2, 1.09/1.15 for MIT and 0.64/0.33 for Golem. Golem’s all-fifty nominal reference reduces error by 0.38, more than the optimized 0.33 despite a smaller return gain. Standard deviations are not paired significance tests.

Complete original PDF block A10.T15
Table 15: Numeric results for the training set comparison. We show the mean ±\pm standard deviation of the return improvement Δ​R\Delta R over the 1010 initial designs per robot, when the target robot is held out from its morphology class training set (Class) or included in the full 50-robot training set (All 50). The last row shows the return of the initial designs.
Original selectable table
Go2 MIT Golem
Method Class All 50 Class All 50 Class All 50
VGDS 24.10±10.9024.10\pm 10.90 25.62±10.94\mathbf{25.62\pm 10.94} 13.75±1.0913.75\pm 1.09 20.47±1.30\mathbf{20.47\pm 1.30} 13.65±9.33\mathbf{13.65\pm 9.33} 5.23±3.44\mathbf{5.23\pm 3.44}
Random 9.90±13.729.90\pm 13.72 8.54±13.228.54\pm 13.22 1.18±3.601.18\pm 3.60 1.88±3.371.88\pm 3.37 0.32±1.460.32\pm 1.46 −0.31±3.61-0.31\pm 3.61
BO 11.47±16.5711.47\pm 16.57 15.04±13.5715.04\pm 13.57 6.61±6.286.61\pm 6.28 8.65±5.908.65\pm 5.90 1.00±0.771.00\pm 0.77 −0.04±4.02-0.04\pm 4.02
CMA-ES 16.03±8.5216.03\pm 8.52 16.84±12.7716.84\pm 12.77 2.17±3.762.17\pm 3.76 7.67±4.087.67\pm 4.08 1.54±1.961.54\pm 1.96 0.01±3.950.01\pm 3.95
TuRBO 13.02±14.4313.02\pm 14.43 15.43±15.1615.43\pm 15.16 1.60±3.171.60\pm 3.17 6.47±5.806.47\pm 5.80 1.70±2.351.70\pm 2.35 −0.11±4.73-0.11\pm 4.73
PSO 11.88±14.3711.88\pm 14.37 14.50±8.1314.50\pm 8.13 3.80±4.213.80\pm 4.21 10.18±7.3510.18\pm 7.35 1.45±2.231.45\pm 2.23 0.40±4.550.40\pm 4.55
CEM 20.51±10.2220.51\pm 10.22 20.24±9.0720.24\pm 9.07 4.47±4.664.47\pm 4.66 12.70±6.1212.70\pm 6.12 1.62±1.621.62\pm 1.62 0.52±2.980.52\pm 2.98
DE 16.39±9.4716.39\pm 9.47 15.25±12.0715.25\pm 12.07 0.88±2.850.88\pm 2.85 12.89±7.0312.89\pm 7.03 0.62±0.800.62\pm 0.80 2.21±3.572.21\pm 3.57
ARS 20.56±10.5120.56\pm 10.51 21.81±10.9621.81\pm 10.96 2.51±4.182.51\pm 4.18 10.40±5.7910.40\pm 5.79 0.65±1.020.65\pm 1.02 1.07±4.131.07\pm 4.13
GC-PFO 24.42±10.67\mathbf{24.42\pm 10.67} 25.01±10.6525.01\pm 10.65 13.82±2.25\mathbf{13.82\pm 2.25} 18.11±3.3818.11\pm 3.38 2.22±1.532.22\pm 1.53 1.65±3.341.65\pm 3.34
freff_{\mathrm{ref}} 23.37±10.8123.37\pm 10.81 23.63±10.6923.63\pm 10.69 14.20±0.8114.20\pm 0.81 17.04±0.9617.04\pm 0.96 0.69±0.620.69\pm 0.62 5.13±3.465.13\pm 3.46
R⁡(finit)R(f_{\mathrm{init}}) 14.46±10.8014.46\pm 10.80 13.70±10.6913.70\pm 10.69 0.32±0.820.32\pm 0.82 0.36±0.960.36\pm 0.96 0.58±0.620.58\pm 0.62 2.47±3.382.47\pm 3.38
Complete original PDF block A10.T16
Table 16: Numeric tracking error results for the training set comparison. We show the mean ±\pm standard deviation of the reduction Δ​ex​y\Delta e_{xy} in absolute xx-yy linear velocity tracking error in m/s over the 1010 initial designs per robot, when the target robot is held out from its morphology class training set (Class) or included in the full 50-robot training set (All 50). The last row shows the tracking error of the initial designs.
Original selectable table
Go2 MIT Golem
Method Class All 50 Class All 50 Class All 50
VGDS 0.19±0.14\mathbf{0.19\pm 0.14} 0.18±0.11\mathbf{0.18\pm 0.11} 1.09±0.58\mathbf{1.09\pm 0.58} 1.15±0.59\mathbf{1.15\pm 0.59} 0.64±0.50\mathbf{0.64\pm 0.50} 0.33±0.50\mathbf{0.33\pm 0.50}
Random 0.13±0.130.13\pm 0.13 0.08±0.070.08\pm 0.07 0.30±0.530.30\pm 0.53 0.42±1.010.42\pm 1.01 0.20±0.640.20\pm 0.64 −0.21±0.90-0.21\pm 0.90
BO 0.14±0.170.14\pm 0.17 0.14±0.110.14\pm 0.11 0.75±0.500.75\pm 0.50 1.00±0.661.00\pm 0.66 0.31±0.500.31\pm 0.50 0.30±0.510.30\pm 0.51
CMA-ES 0.16±0.140.16\pm 0.14 0.14±0.110.14\pm 0.11 0.77±0.660.77\pm 0.66 1.06±0.591.06\pm 0.59 0.34±0.510.34\pm 0.51 −0.23±0.36-0.23\pm 0.36
TuRBO 0.14±0.160.14\pm 0.16 0.13±0.130.13\pm 0.13 0.65±0.740.65\pm 0.74 0.83±0.650.83\pm 0.65 0.11±0.840.11\pm 0.84 0.21±0.290.21\pm 0.29
PSO 0.10±0.200.10\pm 0.20 0.14±0.100.14\pm 0.10 0.82±0.560.82\pm 0.56 0.93±0.630.93\pm 0.63 0.35±0.510.35\pm 0.51 −0.04±0.93-0.04\pm 0.93
CEM 0.17±0.140.17\pm 0.14 0.16±0.110.16\pm 0.11 0.54±0.710.54\pm 0.71 1.08±0.601.08\pm 0.60 0.23±0.680.23\pm 0.68 −0.35±0.62-0.35\pm 0.62
DE 0.16±0.140.16\pm 0.14 0.14±0.110.14\pm 0.11 0.61±0.630.61\pm 0.63 1.09±0.561.09\pm 0.56 0.15±0.230.15\pm 0.23 0.30±0.500.30\pm 0.50
ARS 0.17±0.150.17\pm 0.15 0.16±0.120.16\pm 0.12 0.83±0.560.83\pm 0.56 1.08±0.561.08\pm 0.56 0.07±0.550.07\pm 0.55 0.10±0.290.10\pm 0.29
GC-PFO 0.19±0.14\mathbf{0.19\pm 0.14} 0.18±0.11\mathbf{0.18\pm 0.11} 1.09±0.58\mathbf{1.09\pm 0.58} 1.14±0.581.14\pm 0.58 0.36±0.520.36\pm 0.52 0.08±0.500.08\pm 0.50
freff_{\mathrm{ref}} 0.19±0.140.19\pm 0.14 0.17±0.110.17\pm 0.11 1.09±0.581.09\pm 0.58 1.13±0.591.13\pm 0.59 0.29±0.490.29\pm 0.49 0.38±0.500.38\pm 0.50
ex​y​(finit)e_{xy}(f_{\mathrm{init}}) 0.30±0.140.30\pm 0.14 0.28±0.110.28\pm 0.11 1.37±0.581.37\pm 0.58 1.36±0.591.36\pm 0.59 0.77±0.490.77\pm 0.49 0.76±0.500.76\pm 0.50

Appendix K RL Co-Design Baselines

Five methods are substantially adapted to fixed-topology continuous design, URMA networks and on-policy PPO. Results compare these adaptations of the ideas, not faithful original implementations or their topology-search abilities.

Schaff2019.

Schaff2019 updates a population distribution over normalized designs using rollout returns while training a design-conditioned PPO controller. Environments sample from the current distribution.

Fast Evolutionary Actor-Critic Reinforcement Learning.

Original FEACRL uses off-policy SAC and an action-value critic. The adaptation substitutes PPO and state-value scoring over a start-state buffer, with PSO design selection, changing both learning and scoring.

Transform2Act.

Transform2Act proposes a continuous design update at episode start, followed by its execution policy. Both use PPO; discrete skeleton changes are removed.

BodyGen.

BodyGen retains stage-specific credit assignment and separate design/control critics. Its topology-aware self-attention is omitted in favor of fixed topology and URMA descriptions.

Stackelberg PPO.

Stackelberg PPO treats the design policy as leader and the adapting URMA/PPO controller as follower, within the same fixed-topology continuous space.

Appendix L All 50 Robots Design Results

Figure 12 retains all fifty per-robot results, with ten random starts each. All are training-set robots. Some large humanoids have near-zero gains, attributed to incompletely converged controllers; no held-out-all-fifty generalization claim follows.

Complete original PDF block A12.F12
Figure 12: Design search on all 50 training robots. Per-robot return improvement Δ​R=Rfound−Rinit\Delta R=R_{\mathrm{found}}-R_{\mathrm{init}} of VGDS applied to each robot in the full 50-robot training set, starting from 1010 uniformly sampled random initial designs per robot.

Appendix M Design Analysis Details

Signed RMS summarizes group changes relative to the reference. Selected substitutions recover only part of the gain: Go2 combined groups return 25.1 versus full-design 42.0; MIT gains alone 12.5 versus 31.1; Golem action scale 14.1 versus 20.3. Golem PD gains alone worsen the start to 10.4.

No final coordinate in thirty single-robot designs exceeds |fᵢ|=0.9; average normalized changes are 0.027–0.042. These runs are not simply saturating bounds, without implying optimality. Seventeen partial interventions have overall critic/return Spearman correlation 0.40. Stretched meshes can be rendering artifacts, but Golem still has up to eight self-collisions at the nominal pose. Zero Go2/MIT collisions at one pose does not certify collision-free motion.

Complete original PDF block A13.F13
Figure 13: Grouped design changes for the Unitree Go2 and Golem. Signed RMS changes of the optimized designs relative to the nominal reference, grouped by body part and parameter type. The heatmaps show which design groups are changed most consistently by VGDS.
Complete original PDF block A13.F14
Figure 14: Parameter group evaluations. Selected parameter groups from the optimized design f⋆f^{\star} are copied into the initial design finitf_{\mathrm{init}} and evaluated with the same policy.

Appendix N Design Search Trajectories

Figures 15–17 retain six stages for eight starts per robot, all 144 panels. Designs converge toward similar forms, reducing diversity. Fewer iterations, weaker penalties or lower learning rates are suggested trade-offs without a separate quantitative diversity study.

Complete original figure A14.F15
Figure 15: Design search trajectories for the Unitree Go2 quadruped. The columns show 8 different initial designs, and the rows show the initial design and VGDS at iteration 10, 20, 30, 40, and 50.
Complete original figure A14.F16
Figure 16: Design search trajectories for the MIT Humanoid. The columns show 8 different initial designs, and the rows show the initial design and VGDS at iteration 10, 20, 30, 40, and 50.
Complete original figure A14.F17
Figure 17: Design search trajectories for the Golem hexapod. The columns show 8 different initial designs, and the rows show the initial design and VGDS at iteration 10, 20, 30, 40, and 50.

Appendix O Additional Method Variants

The appendix reports alternatives that did not consistently beat the chosen method, clarifying empirical design choices and failure modes.

Hard trust-region projection.

A hard-radius projection is competitive in some settings, but its radius proved harder to tune than the soft penalty. Both retain reference dependence.

Uncertainty-based trust regions.

Absolute/relative head disagreement and lower-confidence objectives fail as reliable reference-free trust regions: shared-data heads can agree on catastrophic extrapolation. Disagreement is a possible warning, not a safety certificate.

Search dynamics.

Noise, moment resets, norm clipping, cosine schedules, SGD and rate/step sweeps do not consistently improve the selected Adam configuration. More iterations can exploit critic errors further.

Multi-start search.

Nearby starting particles with surrogate-based selection do not consistently improve on one design once the penalty is tuned. This does not establish global optimality of single-start search.

Direction and subspace structure.

Restricting updates to large-gradient coordinates hurts. Removing one dominant direction is mostly harmless, while using only that direction collapses performance, suggesting useful updates are distributed across parameters.

Refreshing the state bank.

Refreshing the bank with current-design rollouts or initial states does not improve results and sometimes worsens them. Overlapping state distributions and a robust policy are the authors’ explanation, not proof of unbiased design-return gradients from fixed states.

Concept demonstration

Open image ↗