Literature
Display
← All papers

HuGo: LLMs as Whole-Body Policy Code Designers for Humanoid Loco-Manipulation

Seoyeon Choi · Shizhao Ye · Nicholas Bui · Aayushi Shrivastava · Kanghyun Ryu · Dhruva Tirumala · Markus Wulfmeier · Negar Mehr

Preprint posted · arXiv (preprint) ↗

Robotics9 min text · CondensedOriginal paper ↗
Contents

Overview

HuGo asks a language model to write and revise executable control code above a frozen humanoid controller. This edition retains all eight original figures, four tables, two equations and Algorithm 1 from arXiv v2, including the opening diagram absent from HTML and the original hardware code diff. Read the results as policy search with rollout-based selection: the deployed code is fast, but obtaining it requires repeated trials and reliable numerical state observations.

Original HuGo framework and hardware examples from PDF Figure 1
Fig. 1: We introduce HuGo, a framework for generating closed-loop policy codes for humanoid loco-manipulation from task descriptions, without expert demonstrations, reference motions, or reward design. (Top left) The generated code outputs a high-level command aʰ at every timestep, consisting of planar locomotion, left and right end-effector positions, and body height, which a frozen low-level whole-body policy converts into joint actions. (Bottom left) The LLM generates the policy code, diagnoses the policy’s rollouts from numerical trajectories and selected video frames, and proposes a code diff, a localized update to the current policy rather than a full regeneration. (Right) Policies generated in simulation transferred zero-shot to hardware for Push Button (top) and Lift Box (bottom).

At a glance

The language model receives task, robot, observation and command descriptions, generates a stateful high-level policy, and improves it through numerical/video rollout feedback. Across five simulated tasks and two low-level controllers, selected policies outperform the tested high-level reinforcement-learning baseline; two additional tasks compare with demonstration-based control. Simulation-generated code transfers to hardware, and four refinement rounds improve one weak Push Button policy from 20% to 90%. No task-specific demonstrations or dense rewards are supplied to HuGo, but the low-level controllers are pretrained and policy selection uses task-success evaluations.

I Introduction

Humanoid tasks require navigation, reaching, contact and correctly timed transitions. HuGo moves this task logic into generated code while reusing a controller that tracks locomotion, hand positions and body height. It supplies no example policies, action sequences or callable task-skill library. The code reads observations at every control step; the language model is queried during generation and refinement. This avoids training a new high-level neural policy for every task, while retaining the costs of repeated rollout evaluation.

II Related Works

The comparison spans language-model optimization, programmatic robot control and hierarchical humanoid learning. HuGo directly edits the high-level controller rather than learning a task policy from a generated reward.

II-A LLM-Guided Optimization

Methods such as Eureka use language models to shape learning objectives, and others optimize programs or evaluate behavior. HuGo uses a numerical/video evaluator to diagnose a rollout and then modifies policy code. Training the low-level controller remains outside this per-task refinement loop.

II-B LLM-Generated Programmatic Robot Policies

Code as Policies and subsequent systems compose supplied perception, planning or skill modules. HuGo receives a low-level command contract and must infer approach geometry, stopping conditions and stage transitions itself. Its generated stateful code executes continually from numerical observations; visual frames support offline diagnosis rather than serving as the runtime policy input.

II-C Humanoid Loco-Manipulation

Hierarchical humanoid systems often obtain high-level behavior through reinforcement learning, retargeted demonstrations, video priors or task-specific optimization. HuGo replaces that high-level learning step with program generation, while retaining an unmodified low-level whole-body controller and its tracking/contact limitations.

III Problem Formulation

A task specifies its natural-language goal, initial-state distribution and binary success predicate. The ten-dimensional command comprises planar base motion/heading, two three-dimensional hand targets and body height; the particular controller defines frames and limits. Observations contain numerical robot and task poses, measured motion and the preceding command. These quantities are assumed available through simulation, tracking or vision; HuGo does not solve raw-image state estimation. The objective is expected rollout success under randomized initial conditions.

Equation 1
ah=[abase,peeL,peeR,zbody]∈ℝ10,a^{h}=\left[a_{\mathrm{base}},p_{\mathrm{ee}}^{\mathrm{L}},p_{\mathrm{ee}}^{\mathrm{R}},z_{\mathrm{body}}\right]\in\mathbb{R}^{10}, (1)

IV Method

The high-level program maps the current observation and its own persistent memory to a command and updated memory. Memory can encode approach/alignment/contact stages and command smoothing. A language model constructs and revises this program outside the control loop, allowing the resulting code to run at the controller’s required frequency.

Equation 2
(ath,mt)=πh​(ot,mt−1).\left(a^{h}_{t},m_{t}\right)=\pi_{h}\left(o_{t},m_{t-1}\right). (2)

IV-A Zero-shot Policy Generation

The fixed prompt describes the task, relevant robot dimensions, observations, accepted commands, coordinate frames and bounds. The initial code must adapt to changing object poses instead of replaying a trajectory. It contains no supplied demonstrations or reference implementation. Plausible geometry may still fail through tracking error, contact dynamics or poor phase transitions, motivating execution-based revision.

IV-B Rollout Evaluation and Feedback

A candidate is scored on M initial states, then one rollout—preferably the first failure—is diagnosed. From sparse numerical observations, internal state and current code, the language model writes an analysis function that scans the complete trajectory. It returns a numerical summary capped at 200 words and task-relevant keyframe indices. Selected keyframes are supplemented with evenly spaced frames to fill a ten-frame budget. Feedback combines those images, trajectory samples and summary. The feedback generator does not receive the candidate’s aggregate success rate; the separate selection loop uses it.

IV-C Iterative Policy Code Refinement

Each refinement depth gives the current policy up to three localized code edits. A strictly higher measured success rate accepts the candidate, clears failed-attempt history and starts the next depth. Otherwise its diff, score and diagnosis remain in context while trying another edit to the same parent. Search stops at 100% success, the depth limit, or three unsuccessful rounds in a depth, returning the current best. History is therefore local, and successful revisions need not escape a poor initial task strategy. Algorithm 1 preserves the exact acceptance and stopping rules.

Algorithm 1 Generation of a Single High-Level Policy
Algorithm 1: original Require/Ensure and 22 numbered lines, with exact indentation

IV-D Best-of- N Selection

Independent initial programs often encode different task strategies, which local edits seldom replace. HuGo runs the complete generation/refinement pipeline N times and selects the highest-scoring result. Best-of-N performance measures this search-and-selection procedure rather than the reliability of a single initial language-model response.

Original complete Figure 2 composite with all panels and labels
Fig. 2: The five simulated loco-manipulation tasks. All are situated in an indoor environment and randomize their initial conditions, so a policy must adapt its navigation path, approach pose, and command timing to each episode.

V Simulation Experiments

Simulation tests randomized navigation/contact tasks, two frozen controller interfaces, generation/refinement variability, selection cost and feedback ablations. Final performance is evaluated separately from the states used for policy refinement and selection.

Low-level Method Low Gate Passing Narrow Gate Passing Push Button Push Box Lift Box
rl_isaac High-level RL 11.33±1.7011.33\pm 1.70 0.0±0.00.0\pm 0.0 7.33±1.707.33\pm 1.70 1.67±1.251.67\pm 1.25 0.0±0.00.0\pm 0.0
HuGo (no refinement) 100.00±0.00100.00\pm 0.00 67.15±14.1367.15\pm 14.13 73.92±7.5673.92\pm 7.56 49.31±17.4949.31\pm 17.49 44.62±25.2244.62\pm 25.22
HuGo 100.00±0.00\boldsymbol{100.00\pm 0.00} 96.20±1.31\boldsymbol{96.20\pm 1.31} 79.97±1.68\boldsymbol{79.97\pm 1.68} 89.33±11.37\boldsymbol{89.33\pm 11.37} 75.92±16.44\boldsymbol{75.92\pm 16.44}
sonic HuGo (no refinement) - 65.93±21.9865.93\pm 21.98 48.70±16.4348.70\pm 16.43 46.30±25.8646.30\pm 25.86 11.00±3.7911.00\pm 3.79
HuGo - 97.54±1.97\boldsymbol{97.54\pm 1.97} 85.78±7.89\boldsymbol{85.78\pm 7.89} 61.79±12.18\boldsymbol{61.79\pm 12.18} 24.91±6.12\boldsymbol{24.91\pm 6.12}
TABLE I: Success rates (%) on the five-task suite with best-of-1010 selection. Reported success rates are evaluated on 100100 held-out initial states not used for policy refinement or best-of-N selection. The same held-out initial states are used for all methods. HuGo (no refinement) selects among the same policies before refinement. sonic is not evaluated on LGP as it cannot change its height while walking.

V-A Experiment Setup

The five tasks are low-gate passage, sideways narrow-gate passage, button pressing, box pushing to within 10 cm for one second, and lifting a box above 1 m. Indoor obstacles and randomized robot/object/goal poses require closed-loop adaptation. The rl_isaac controller tracks base velocity/yaw rate, torso-frame hands and torso height; SONIC uses planar velocity/yaw heading, pelvis-frame hands and pelvis height. SONIC cannot change height while walking, so low-gate passage is excluded. The high-level PPO baseline receives the same interface and 384 million environment steps with hand-designed task rewards. HDMI supplies a human-video demonstration baseline on door passage and bread-box transport; those tasks fix relative initial poses and HuGo’s bread-box mass is reduced to a supported payload. Experiments use a Unitree G1 in IsaacSim, one L40S GPU and the reported gpt-5.4 model. Code runs at 50 Hz with rl_isaac and 10 Hz with SONIC. Thirty independently generated/refined policies use 100 fixed selection states, up to seven depths and three rounds per depth. Best-of-ten estimates average 2,000 random ten-policy subsets, with final testing on 100 held-out initial states.

PDH CPB
HDMI [9] 99.95±2.2199.95\pm 2.21 98.4±12.698.4\pm 12.6
HuGo 100.0±0.00100.0\pm 0.00 81.25±26.2381.25\pm 26.23
TABLE II: Success rates (%) on two HDMI tasks. HDMI trains a policy per task from a human demonstration, while our policies are generated from a task description without demonstrations. HuGo is reported with best-of-1010 selection.
Original figure
Fig. 3: Best-of-1010 success rate versus refinement depth, averaged over low-level policies. Depth 00 corresponds to the policies before refinement.
Original figure
Original figure
Fig. 4: Best-of-NN success rate against the cost of obtaining it. Left: API cost, averaged over the five tasks and both low-level policies, where error bars denote the standard deviation across runs and the red marker indicates the operating point N=10N=10. Right: wall-clock time, shown separately for each low-level policy.
NGP PBT PBX LBX Avg.
Zero-shot 0.00.0 1.01.0 0.00.0 0.00.0 0.250.25
Text Only 47.0±38.847.0\pm 38.8 53.2±23.953.2\pm 23.9 19.6±36.719.6\pm 36.7 1.4±2.31.4\pm 2.3 30.330.3
Even Only 52.4±42.952.4\pm 42.9 55.4±29.355.4\pm 29.3 1.8±1.31.8\pm 1.3 13.0±18.213.0\pm 18.2 30.630.6
Keyframe Only 56.6±46.356.6\pm 46.3 45.6±28.345.6\pm 28.3 5.4±6.25.4\pm 6.2 20.2±16.020.2\pm 16.0 32.032.0
Ours 56.8±46.4\boldsymbol{56.8\pm 46.4} 75.8±4.6\boldsymbol{75.8\pm 4.6} 42.4±34.1\boldsymbol{42.4\pm 34.1} 35.6±31.9\boldsymbol{35.6\pm 31.9} 52.6\boldsymbol{52.6}

TABLE III: Success rates (%) after refinement when the rollout evaluation is given different information, mean ±\pm standard deviation over five runs per task using rl_isaac. All variants start from the same zero-shot policies, shown in the first row.
Original complete Figure 5 composite with all panels and labels
Fig. 5: PBT on hardware, all from the same policy code. The button position and facing direction determine which side the policy approaches from and which end-effector it presses with.

V-B Main Results

With rl_isaac, selected/refined success is 100.0% low gate, 96.2% narrow gate, 80.0% button, 89.3% push box and 75.9% lift box. The tested high-level RL baseline remains at 0–11.3% across these tasks. SONIC reaches 97.5%, 85.8%, 61.8% and 24.9% on the four applicable tasks. Refinement improves push-box success with rl_isaac by about 40 percentage points; most gains occur early. Lift-box diagnosis is a clear limitation: hands approach directly through the box sides, and SONIC’s roughly 4 cm tracking error worsens collisions. A human-supplied geometry hint raises SONIC lifting to 72.3%, showing that the interface can express a correction that autonomous diagnosis misses. Both controllers also lose grasped boxes through inadequate sustained contact force. On HDMI tasks, HuGo achieves 100% door passage versus 99.95%, but 81.25% bread-box transport versus 98.4%; the reduced box mass and different training inputs qualify this comparison.

V-C Variation of Performance

A single generated/refined policy averages 43.9% success with standard deviation 23.4 across the reported task/controller aggregate. For box pushing, one strategy continually aligns robot–box–goal, while another changes pushing faces and may run out of time. Variation reflects alternative program structures, so selecting among independent starts complements local code repair. It also means one arbitrary generated policy should not be treated as dependable.

PBT LBX
Policy Sim Real Sim Real
1 9090 93.393.3 7878 66.766.7
2 9090 20.020.0 7070 20.020.0
3 7979 73.373.3 6464 6.76.7
TABLE IV: Success rates (%) of the three policy codes per task with the highest simulation success rate, evaluated in simulation (100100 episodes) and on hardware (1515 trials).
Original complete Figure 6 composite with all panels and labels
Fig. 6: LBX on hardware, all from the same policy code. The policy sets its body height and end-effector targets from the height of the box rather than replaying a fixed motion.

V-D Cost of Reliability

Increasing selection from one to ten policies raises reported mean success from 43.9% to 79.1% and reduces standard deviation from 23.4 to 6.6. The paper reports about $1.10 in model API calls per generated/refined policy, or $11 for best-of-ten, with diminishing gains beyond roughly eight starts. Those are reported experimental costs, not a general deployment price. Rollout simulation dominates wall time: rl_isaac supports batches whereas SONIC evaluates environments serially. Search takes hours in this setup; the tested RL baseline takes over two days for one reward-specific training run.

V-E Ablation Studies

The feedback ablation starts from the same near-zero-success policies on four tasks, excluding already-solved low-gate passage, and repeats refinement five times using rl_isaac. Combined numerical, keyframe and evenly spaced visual feedback averages 52.6%, versus 30.3% with text alone, 30.6% with evenly spaced frames and 32.0% with selected keyframes alone. Removing images costs 22.3 percentage points, about 42% relatively. These large-variance experiments support complementary temporal coverage and contact imagery in this setup; they do not show that images are universally necessary for every policy or task.

VI Hardware Experiments

Hardware studies separate unchanged transfer from additional rollout-driven editing. They use tracked numerical state and a matching low-level interface; transfer is not end-to-end visual perception.

VI-A Hardware Setup

The physical Unitree G1 uses the same SONIC controller and observation/command contract as simulation. A motion-capture system supplies state, allowing simulation-generated high-level code to execute unchanged. Deployment still relies on this measurement infrastructure and the pretrained controller.

VI-B Zero-Shot Transfer of Simulation-Generated Policies

Three top simulation policies per task are each tested in 15 physical trials on button pressing and box lifting, using matched initial conditions. Lifting policies include the geometry hint. The best button policy succeeds in 93.3% of trials versus 90% in simulation; the best lifting policy reaches 66.7% versus 78%. Transfer varies sharply: the other button policies reach 20.0% and 73.3%, and the other lifting policies 20.0% and 6.7%. Small velocity tracking and contact-force mismatch explain observed losses. Figures 5–6 show selected initial conditions, while Table IV preserves every policy’s result; the best-policy numbers do not characterize all generated programs.

VI-C Refinement on Hardware

The worst-transferring button policy starts at 20% hardware success despite 90% in simulation. With ten physical trials per candidate, four attempted edits across three accepted depths produce 50%, 60% and 90%. Feedback identifies hand underreach after successful navigation. One revision adds a 3 cm inward press margin; a later one increases the margin and press duration. The same loop changes code from real rollouts without retraining the controller or collecting expert demonstrations. This is a small, selected-policy adaptation experiment; the paper does not report a separate held-out hardware test after refinement.

Original figure
Fig. 7: Success rate at each refinement depth on hardware, starting from the policy that transfers worst. The simulation success rate of the same policy is shown for reference.

VII Conclusion

Generated closed-loop programs can reuse whole-body skills while avoiding new task-specific demonstrations or dense rewards. Their limits include low-level contact control, reliable measured state, search variability and incorrect causal diagnosis by the language model. The lifting geometry hint exposes dependence on external guidance in a difficult case. Proposed extensions include cross-task memory and using successful programs as data or initialization for learned visual/RL policies.

Refer to caption
Fig. 8: Example of one refinement depth on hardware, generated by the LLM without any human feedback. Top: executions at refinement depths 11 to 33. Bottom: at depth 11, the rollout feedback attributes the failure to the end-effector under-reaching the button, and the resulting code change introduces an explicit inward margin to compensate for the undershoot.

ACKNOWLEDGMENT

The authors acknowledge manuscript feedback from Lasse Peters and Hongrui Zhao, and feedback and the G1 end-effector design from Joohwan Seo.

Concept demonstration

Open image ↗