Overview
HuGo asks a language model to write and revise executable control code above a frozen humanoid controller. This edition retains all eight original figures, four tables, two equations and Algorithm 1 from arXiv v2, including the opening diagram absent from HTML and the original hardware code diff. Read the results as policy search with rollout-based selection: the deployed code is fast, but obtaining it requires repeated trials and reliable numerical state observations.
At a glance
The language model receives task, robot, observation and command descriptions, generates a stateful high-level policy, and improves it through numerical/video rollout feedback. Across five simulated tasks and two low-level controllers, selected policies outperform the tested high-level reinforcement-learning baseline; two additional tasks compare with demonstration-based control. Simulation-generated code transfers to hardware, and four refinement rounds improve one weak Push Button policy from 20% to 90%. No task-specific demonstrations or dense rewards are supplied to HuGo, but the low-level controllers are pretrained and policy selection uses task-success evaluations.
I Introduction
Humanoid tasks require navigation, reaching, contact and correctly timed transitions. HuGo moves this task logic into generated code while reusing a controller that tracks locomotion, hand positions and body height. It supplies no example policies, action sequences or callable task-skill library. The code reads observations at every control step; the language model is queried during generation and refinement. This avoids training a new high-level neural policy for every task, while retaining the costs of repeated rollout evaluation.
II Related Works
The comparison spans language-model optimization, programmatic robot control and hierarchical humanoid learning. HuGo directly edits the high-level controller rather than learning a task policy from a generated reward.
II-A LLM-Guided Optimization
Methods such as Eureka use language models to shape learning objectives, and others optimize programs or evaluate behavior. HuGo uses a numerical/video evaluator to diagnose a rollout and then modifies policy code. Training the low-level controller remains outside this per-task refinement loop.
II-B LLM-Generated Programmatic Robot Policies
Code as Policies and subsequent systems compose supplied perception, planning or skill modules. HuGo receives a low-level command contract and must infer approach geometry, stopping conditions and stage transitions itself. Its generated stateful code executes continually from numerical observations; visual frames support offline diagnosis rather than serving as the runtime policy input.
II-C Humanoid Loco-Manipulation
Hierarchical humanoid systems often obtain high-level behavior through reinforcement learning, retargeted demonstrations, video priors or task-specific optimization. HuGo replaces that high-level learning step with program generation, while retaining an unmodified low-level whole-body controller and its tracking/contact limitations.
III Problem Formulation
A task specifies its natural-language goal, initial-state distribution and binary success predicate. The ten-dimensional command comprises planar base motion/heading, two three-dimensional hand targets and body height; the particular controller defines frames and limits. Observations contain numerical robot and task poses, measured motion and the preceding command. These quantities are assumed available through simulation, tracking or vision; HuGo does not solve raw-image state estimation. The objective is expected rollout success under randomized initial conditions.
Equation 1
| (1) |
IV Method
The high-level program maps the current observation and its own persistent memory to a command and updated memory. Memory can encode approach/alignment/contact stages and command smoothing. A language model constructs and revises this program outside the control loop, allowing the resulting code to run at the controller’s required frequency.
Equation 2
| (2) |
IV-A Zero-shot Policy Generation
The fixed prompt describes the task, relevant robot dimensions, observations, accepted commands, coordinate frames and bounds. The initial code must adapt to changing object poses instead of replaying a trajectory. It contains no supplied demonstrations or reference implementation. Plausible geometry may still fail through tracking error, contact dynamics or poor phase transitions, motivating execution-based revision.
IV-B Rollout Evaluation and Feedback
A candidate is scored on M initial states, then one rollout—preferably the first failure—is diagnosed. From sparse numerical observations, internal state and current code, the language model writes an analysis function that scans the complete trajectory. It returns a numerical summary capped at 200 words and task-relevant keyframe indices. Selected keyframes are supplemented with evenly spaced frames to fill a ten-frame budget. Feedback combines those images, trajectory samples and summary. The feedback generator does not receive the candidate’s aggregate success rate; the separate selection loop uses it.
IV-C Iterative Policy Code Refinement
Each refinement depth gives the current policy up to three localized code edits. A strictly higher measured success rate accepts the candidate, clears failed-attempt history and starts the next depth. Otherwise its diff, score and diagnosis remain in context while trying another edit to the same parent. Search stops at 100% success, the depth limit, or three unsuccessful rounds in a depth, returning the current best. History is therefore local, and successful revisions need not escape a poor initial task strategy. Algorithm 1 preserves the exact acceptance and stopping rules.
IV-D Best-of- N Selection
Independent initial programs often encode different task strategies, which local edits seldom replace. HuGo runs the complete generation/refinement pipeline N times and selects the highest-scoring result. Best-of-N performance measures this search-and-selection procedure rather than the reliability of a single initial language-model response.
V Simulation Experiments
Simulation tests randomized navigation/contact tasks, two frozen controller interfaces, generation/refinement variability, selection cost and feedback ablations. Final performance is evaluated separately from the states used for policy refinement and selection.
| Low-level | Method | Low Gate Passing | Narrow Gate Passing | Push Button | Push Box | Lift Box |
| rl_isaac | High-level RL | |||||
| HuGo (no refinement) | ||||||
| HuGo | ||||||
| sonic | HuGo (no refinement) | - | ||||
| HuGo | - |
V-A Experiment Setup
The five tasks are low-gate passage, sideways narrow-gate passage, button pressing, box pushing to within 10 cm for one second, and lifting a box above 1 m. Indoor obstacles and randomized robot/object/goal poses require closed-loop adaptation. The rl_isaac controller tracks base velocity/yaw rate, torso-frame hands and torso height; SONIC uses planar velocity/yaw heading, pelvis-frame hands and pelvis height. SONIC cannot change height while walking, so low-gate passage is excluded. The high-level PPO baseline receives the same interface and 384 million environment steps with hand-designed task rewards. HDMI supplies a human-video demonstration baseline on door passage and bread-box transport; those tasks fix relative initial poses and HuGo’s bread-box mass is reduced to a supported payload. Experiments use a Unitree G1 in IsaacSim, one L40S GPU and the reported gpt-5.4 model. Code runs at 50 Hz with rl_isaac and 10 Hz with SONIC. Thirty independently generated/refined policies use 100 fixed selection states, up to seven depths and three rounds per depth. Best-of-ten estimates average 2,000 random ten-policy subsets, with final testing on 100 held-out initial states.
| PDH | CPB | |
| HDMI [9] | ||
| HuGo |
| NGP | PBT | PBX | LBX | Avg. | |
| Zero-shot | |||||
| Text Only | |||||
| Even Only | |||||
| Keyframe Only | |||||
| Ours |
V-B Main Results
With rl_isaac, selected/refined success is 100.0% low gate, 96.2% narrow gate, 80.0% button, 89.3% push box and 75.9% lift box. The tested high-level RL baseline remains at 0–11.3% across these tasks. SONIC reaches 97.5%, 85.8%, 61.8% and 24.9% on the four applicable tasks. Refinement improves push-box success with rl_isaac by about 40 percentage points; most gains occur early. Lift-box diagnosis is a clear limitation: hands approach directly through the box sides, and SONIC’s roughly 4 cm tracking error worsens collisions. A human-supplied geometry hint raises SONIC lifting to 72.3%, showing that the interface can express a correction that autonomous diagnosis misses. Both controllers also lose grasped boxes through inadequate sustained contact force. On HDMI tasks, HuGo achieves 100% door passage versus 99.95%, but 81.25% bread-box transport versus 98.4%; the reduced box mass and different training inputs qualify this comparison.
V-C Variation of Performance
A single generated/refined policy averages 43.9% success with standard deviation 23.4 across the reported task/controller aggregate. For box pushing, one strategy continually aligns robot–box–goal, while another changes pushing faces and may run out of time. Variation reflects alternative program structures, so selecting among independent starts complements local code repair. It also means one arbitrary generated policy should not be treated as dependable.
| PBT | LBX | |||
| Policy | Sim | Real | Sim | Real |
| 1 | ||||
| 2 | ||||
| 3 | ||||
V-D Cost of Reliability
Increasing selection from one to ten policies raises reported mean success from 43.9% to 79.1% and reduces standard deviation from 23.4 to 6.6. The paper reports about $1.10 in model API calls per generated/refined policy, or $11 for best-of-ten, with diminishing gains beyond roughly eight starts. Those are reported experimental costs, not a general deployment price. Rollout simulation dominates wall time: rl_isaac supports batches whereas SONIC evaluates environments serially. Search takes hours in this setup; the tested RL baseline takes over two days for one reward-specific training run.
V-E Ablation Studies
The feedback ablation starts from the same near-zero-success policies on four tasks, excluding already-solved low-gate passage, and repeats refinement five times using rl_isaac. Combined numerical, keyframe and evenly spaced visual feedback averages 52.6%, versus 30.3% with text alone, 30.6% with evenly spaced frames and 32.0% with selected keyframes alone. Removing images costs 22.3 percentage points, about 42% relatively. These large-variance experiments support complementary temporal coverage and contact imagery in this setup; they do not show that images are universally necessary for every policy or task.
VI Hardware Experiments
Hardware studies separate unchanged transfer from additional rollout-driven editing. They use tracked numerical state and a matching low-level interface; transfer is not end-to-end visual perception.
VI-A Hardware Setup
The physical Unitree G1 uses the same SONIC controller and observation/command contract as simulation. A motion-capture system supplies state, allowing simulation-generated high-level code to execute unchanged. Deployment still relies on this measurement infrastructure and the pretrained controller.
VI-B Zero-Shot Transfer of Simulation-Generated Policies
Three top simulation policies per task are each tested in 15 physical trials on button pressing and box lifting, using matched initial conditions. Lifting policies include the geometry hint. The best button policy succeeds in 93.3% of trials versus 90% in simulation; the best lifting policy reaches 66.7% versus 78%. Transfer varies sharply: the other button policies reach 20.0% and 73.3%, and the other lifting policies 20.0% and 6.7%. Small velocity tracking and contact-force mismatch explain observed losses. Figures 5–6 show selected initial conditions, while Table IV preserves every policy’s result; the best-policy numbers do not characterize all generated programs.
VI-C Refinement on Hardware
The worst-transferring button policy starts at 20% hardware success despite 90% in simulation. With ten physical trials per candidate, four attempted edits across three accepted depths produce 50%, 60% and 90%. Feedback identifies hand underreach after successful navigation. One revision adds a 3 cm inward press margin; a later one increases the margin and press duration. The same loop changes code from real rollouts without retraining the controller or collecting expert demonstrations. This is a small, selected-policy adaptation experiment; the paper does not report a separate held-out hardware test after refinement.
VII Conclusion
Generated closed-loop programs can reuse whole-body skills while avoiding new task-specific demonstrations or dense rewards. Their limits include low-level contact control, reliable measured state, search variability and incorrect causal diagnosis by the language model. The lifting geometry hint exposes dependence on external guidance in a difficult case. Proposed extensions include cross-task memory and using successful programs as data or initialization for learned visual/RL policies.
ACKNOWLEDGMENT
The authors acknowledge manuscript feedback from Lasse Peters and Hongrui Zhao, and feedback and the G1 end-effector design from Joohwan Seo.