Overview
This edition follows the 24-page arXiv v2 dated 1 October 2026, retaining five numbered figures, six tables, all 17 numbered equations and 55 references. Native PDF restorations preserve figure layers, small labels and every rollout prompt; the original title emblem is also retained. TANGO predicts whole-body reference motion and relies on a pretrained low-level tracker for execution. The physical results are 15 trials per setting, with explicit traversal instructions in cluttered scenes; they do not establish collision-free operation.
![[Uncaptioned image]](https://storage.literature.yihao.one/media/sha256/b1/b1912e7555bcab2b0e49388093587b07e9d0798e1c029ff5042985484a92a388.webp)
At a glance
TANGO learns language-guided humanoid navigation with whole-body obstacle negotiation instead of limiting the learned action to planar waypoints. Simulation supplies edited and tracker-verified motions for training a vision-language model and flow-matching action expert. A Unitree G1 executes predicted references through a motion tracker without real-world navigation fine-tuning. The system improves the reported navigation and cluttered-traversal comparisons, while remaining dependent on tracking capability and visual observability.
1 Introduction
Whether a humanoid can traverse a route depends on arm, torso and leg configuration as well as its base path. TANGO moves this coordination into a learned whole-body reference policy. Its offline Plan–Edit–Track pipeline supplies obstacle-aware supervision; its deployed system still includes the SONIC tracker and real-time chunking. The contribution is a unified semantic/reference-motion model, not removal of low-level control or a proof of safe navigation.
2 Related Works
Most vision-language navigation models predict planar motion and delegate embodiment details. Humanoid traversal policies add geometry-aware control but often address narrower scenarios. TANGO combines a navigation-pretrained backbone with direct whole-body motion prediction and an off-the-shelf tracker. Unlike methods that separate upper-body actions from lower-body commands or learn task-specific latent decoders, it supervises joint-space references; the experiments test one humanoid platform and two tracker choices, not arbitrary-robot transfer.
3 Method
The system has three stages: synthesize task-aligned whole-body data offline, jointly train a vision-language backbone and action expert, then stream reference chunks to a general-purpose tracker.
3.1 Problem Formulation
The observation includes instruction, a temporal history of front/downward RGB views and current joint-angle proprioception. Each output action contains 29 desired joint angles plus a six-dimensional base-rotation representation. The model predicts a horizon of these references for the tracker. Thus the “29-DoF” headline describes joint control; it is not the total encoded action dimension, and RGB-only geometric perception still accompanies proprioceptive feedback.
3.2 Simulation Data Generation
Filtering 263 VLNVerse and 1,000 SAGE-3D scenes leaves 205 and 373 for augmentation. Targeted side, floor and overhead obstacles induce sidling, stepping and crouching. PET plans an obstacle-penalized planar path, inserts sideways headings in narrow corridors, generates a gait with SONIC, then edits links and footfalls for 3D clearance. Closed-loop tracking discards infeasible/colliding trajectories. Accepted edited references, rather than noisier tracked motion, supervise the policy; observations and instructions are regenerated. The final 64,633 trajectories cost 211 RTX PRO 6000 GPU-hours for generation/rendering, excluding VLA training.
3.3 TANGO Architecture and Training
System-2 uses Qwen2.5-VL-7B initialized from InternVLA-N1. Front/downward images are stacked; history frames are probability-sampled and spatially pooled, with more detail allocated to recent views. System-1 is a flow-matching multimodal diffusion transformer conditioned on the visual-language context and proprioception. Its training target adds relative base yaw and auxiliary planar displacement/heading deltas to stabilize motion learning.
Training-time real-time chunking inpaints a future horizon conditioned on a random committed-action prefix, promoting continuity across updates. Recovered references feed system-0, SONIC. A text branch co-trains video question answering to preserve general knowledge. The joint loss adds flow matching with weight 20 to text cross-entropy; training uses one epoch at learning rate 10⁻⁵.
3.4 Deployment
Simulation separates physics and rendering: MuJoCo runs the tracker and robot dynamics, while an IsaacSim digital twin is teleported to the resulting pose for camera rendering. Physical deployment places the VLA on an RTX PRO 6000 server and SONIC on an onboard Jetson Orin NX. Front/downward cameras and proprioception stream over the network. The learned policy is therefore physically evaluated through a tracker even though its rendering twin uses teleportation.
4 Experiments
The experiments separately test conventional VLN, augmented obstacle traversal, physical transfer and component ablations. These protocols differ in physical execution, sensors, training data and metric definitions, so their rows are not interchangeable.
Methods
Low-level
Control
Val Seen
Val Unseen
NE
OSR
SR
SPL
NE
OSR
SR
SPL
CMA [52]
✗
5.36
59.81
37.35
33.36
5.16
62.79
31.15
27.92
RDP [27]
✗
4.02
68.09
47.28
41.69
3.75
71.93
48.60
42.72
Seq2Seq [52]
✗
4.78
44.68
32.62
30.39
4.36
49.58
35.03
33.37
HNR [53]
✗
–
–
36.34
32.10
–
–
32.95
29.56
InternVLA-N1 [10]
✗
3.91
62.50
51.56
34.37
4.09
64.13
45.56
34.98
Uni-NaVid [47]
✗
4.31
60.62
51.88
39.72
3.97
62.50
45.00
39.42
Ours
✓
3.90
70.31
54.69
40.18
3.72
71.07
52.89
40.18
4.1 VLN Performance
The standard benchmark trains on 3,963 trajectories and evaluates 423 seen and 825 unseen episodes. TANGO reaches 54.69%/52.89% success and 3.90/3.72 m final navigation error. Baselines use teleportation, while TANGO uses physical tracking; this is an explicit protocol difference. TANGO leads reported success and final error, but RDP has higher SPL on both splits and higher unseen oracle success. The source attributes reduced path efficiency to conservative tracking; this is an interpretation rather than an isolated causal test.
4.2 Cluttered Environment Traversal Performance
In augmented unseen scenes, modular InternVLA-N1 baselines execute through Unitree’s controller/MPC or HumanoidPF, which additionally receives LiDAR. TANGO uses RGB scene observations and proprioception. Against fine-tuned InternVLA-N1–HumanoidPF, success rises 41.88→43.75%, SPL 29.49→31.83, and episode collision rate falls 15.81→9.90%. These comparisons support the evaluated integrated reference policy; different inputs and executors remain part of the system comparison.
| Methods | Augmented Val Unseen | |||
|---|---|---|---|---|
| NE | SR | SPL | CR | |
| InternVLA-N1 (zero-shot)† | 5.34 | 26.67 | 11.92 | 19.03 |
| InternVLA-N1 (zero-shot)‡ | 3.19 | 35.29 | 24.35 | 16.11 |
| InternVLA-N1 + Unitree WBC | 4.57 | 35.00 | 17.38 | 17.49 |
| InternVLA-N1 + HumanoidPF | 4.04 | 41.88 | 29.49 | 15.81 |
| Ours | 4.01 | 43.75 | 31.83 | 9.90 |
† Equipped with Unitree low-level controller and MPC. ‡ Tracked by HumanoidPF generalist policy
Method
Short-horizon
2D Env.
Long-horizon
2D Env.
Cluttered
3D Env.
SR
Coll.
SR
Coll.
SR
Coll.
InternVLA-N1†
11/15
1.40
6/15
3.47
6/15
1.93
Ours
12/15
0.40
8/15
1.07
10/15
0.73
† Equipped with Unitree WBC.
4.3 Real-World Experiment
Hardware tests use three scenes with five trials each for short routes (~10 m), longer routes (~30 m) and cluttered routes with one difficult obstacle. TANGO succeeds in 12/15, 8/15 and 10/15 trials versus 11/15, 6/15 and 6/15 for InternVLA-N1–Unitree. Mean collisions per trial are 0.40/1.07/0.73 versus 1.40/3.47/1.93. These are counts, not episode collision percentages. Cluttered instructions explicitly request stepping, sidling or ducking; the qualitative rollouts retain those prompts. Transfer uses no real navigation fine-tuning, but relies on pretrained model/tracker knowledge.
4.4 Ablation Studies
On original unseen scenes, the planar Ours-2D model drops from 45.74% success under teleportation to 26.67% under physical control; TANGO reaches 52.89%. Zero-shot InternVLA-N1 provides context but does not isolate representation from training differences. In augmented scenes, removing chunk continuity drops success 43.75→10.94%; removing motion editing gives 36.25% success and raises collisions 9.90→20.60%. Replacing SONIC with ScaleBFM yields 40.94% success, 29.65 SPL and a slightly lower 9.10% collision rate. This demonstrates a second compatible tracker, with tradeoffs across metrics.
Method
Low-level
control
Val Unseen
SR
SPL
InternVLA-N1
✗
43.69
35.74
Ours-2D
✗
45.74
35.44
InternVLA-N1†
✓
42.37
22.50
Ours-2D†
✓
26.67
8.34
Ours
✓
52.89
40.18
† Equipped with Unitree low-level controller and MPC.
Method
Augmented Val Unseen
SR
SPL
CR
w/o RTC
10.94
10.94
14.60
w/o Motion Editing
36.25
30.36
20.60
SONIC ScaleBFM
40.94
29.65
9.10
Ours
43.75
31.83
9.90
5 Conclusion
Whole-body references, obstacle-aware supervision and chunk continuity improve the tested humanoid navigation systems. Tracking remains a limiting component, particularly for harder behaviors such as stairs. RGB perception can be ambiguous in clutter or poor light; depth or LiDAR integration is future work. Results on small physical trial sets and behavior-explicit commands support feasibility, not unrestricted reliability or broad morphology generalization.
6 Training Details
Training oversamples large turns, sideways motion, squats, strides and mixed behaviors, especially rarer stride/sidle segments. Motion-conditioned losses emphasize the joint and body-frame dimensions relevant to each behavior, rather than treating every action coordinate uniformly. Full-parameter training uses 16 nodes of eight A100 GPUs for about seven hours: 896 GPU-hours, separate from the 211 GPU-hours of data generation/rendering.
7 Deployment Details
The server receives front/downward RealSense images and proprioception with approximately 20 ms communication latency. Inference runs every 0.5 s, with 15 actions per execution segment at 30 Hz; references are resampled to 50 Hz, and low-level tracking runs near 200 Hz onboard. A control panel sends instructions and monitors observations/motion. The communication delay is not the total inference-to-actuation latency.
8 Environment Augmentation and Motion Generation Details
The supplementary pipeline specifies how obstacles, instructions, reference motion and dynamically checked demonstrations are generated. Those offline geometry and path priors are richer than the deployed RGB/proprioceptive observations.
8.1 Environment Augmentation Details
Obstacles are placed along valid paths and aligned with local travel direction, excluding start/goal regions and protecting unrelated routes. Low objects induce strides, paired side objects induce sidling and overhead geometry induces squats. The scene occupancy is updated consistently. After quality filtering, obstacle types comprise 44% stride, 15% sidle and 41% squat. Gemini augments instructions with the obstacle and required behavior; the authors’ observation that such prompts help learning is not a controlled language ablation. Table 6 retains original/augmented dataset counts and separate PET/rendering costs.
|
Stride |
![]() |
![]() |
![]() |
![]() |
![]() |
|---|---|---|---|---|---|
|
Sidle |
![]() |
![]() |
![]() |
![]() |
![]() |
|
Squat |
![]() |
![]() |
![]() |
![]() |
![]() |
| Dataset | Trajectories | GPU-hours | ||
|---|---|---|---|---|
| PET | Rendering | Total | ||
| VLNVerse (original) | 3,478 | 3 | 8 | 11 |
| VLNVerse (augmented) | 3,784 | 6 | 17 | 23 |
| SAGE-3D (original) | 28,678 | 20 | 45 | 65 |
| SAGE-3D (augmented) | 28,693 | 57 | 55 | 112 |
| Total | 64,633 | 86 | 125 | 211 |
8.2 Motion Generation Details
Plan. An A* path uses an exponentially decaying obstacle-clearance penalty. A width rule rotates body heading by ±90° through narrow passages, choosing the wider leading-shoulder side. Finite-differenced forward/lateral/yaw commands drive SONIC, whose prior uses about 800 hours of motion capture. A PD correction reduces path drift; this initial gait remains above-floor obstacle-agnostic.
Edit. A 3D signed-distance field, with exact box geometry for thin bars, supplies guidance only inside an approximately 0.5 m path corridor. Repulsion plus path tangent becomes capped, filtered soft link-displacement tasks in IK, balancing posture, foot contacts and center of mass. It is not a hard collision constraint. Squats use vertical forces, forward clearance probes and a virtual head barrier, then coordinate waist/hip/leg posture and center-of-mass lowering; the source describes waist-only, center-of-mass-drop and full-squat presets. Sidling combines the rewritten heading with lateral link guidance.
For strides, a footprint-aware scan retargets footfalls beyond steppable bars while respecting reach limits and preserving gait phase. Taller obstacles do not trigger step-over. An asymmetric early-peaking clearance arc protects the trailing foot; its height is solved across sampled swing phases. Sequential obstacles use short foothold sequences. Smoothed root and center-of-mass redirection prevent an in-place lunge when support moves forward. Each frame first projects lower-body contact/support targets, then solves compliant whole-body IK with strong foot anchors.
Track. Edited kinematic references pass through closed-loop SONIC in MuJoCo to check feasible execution and filter collisions. Generous landing margins compensate for known tracker undershoot. Passing this finite simulation filter validates those generated trajectories; it does not certify all future learned rollouts.
Equations 1–6
| (1) |
| (2) |
| (3) |
| (4) |
| (5) |
| (6) | ||||
9 Experiment Metrics
Navigation metrics use the executed robot-base trajectory and geodesic goal distance, with a success threshold of three meters. Path length and shortest-path distance support efficiency weighting. The threshold assesses reaching a goal neighborhood, not precise stopping or collision avoidance.
Equations 7–9
| (7) |
| (8) |
| (9) |
Navigation Error (NE).
Navigation error is mean geodesic distance from the final stopping point to the target. Lower values indicate closer final positions.
Equation 10
| (10) |
Success Rate (SR).
Success requires stopping within three meters of the goal. Simulation reports a percentage; physical tables show successes out of 15 trials. This position criterion is separate from the collision metric.
Equations 11–12
| (11) |
| (12) |
Oracle Success Rate (OSR).
Oracle success counts whether any point along the executed trajectory entered the three-meter goal region, even if the robot later moved away. It measures reaching the neighborhood rather than stopping there.
Equations 13–14
| (13) |
| (14) |
Success weighted by Path Length (SPL).
SPL multiplies each successful episode by shortest-path distance divided by the larger of actual and shortest path length. Failures contribute zero, and unnecessarily long successful paths are penalized.
Equation 15
| (15) |
Collision Rate (CR).
Simulation collision rate is the percentage of episodes with at least one collision. It does not count how many collisions occurred within each episode or their severity.
Equations 16–17
| (16) |
| (17) |
Mean collisions in real-world trials.
Physical collision reporting averages the number of observed collisions across 15 trials. Multiple events in one trial count separately, so values such as 1.07 are mean counts rather than 107% collision rates.














