Literature
Display
← All papers

Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization

Hao Luo · Ye Wang · Wanpeng Zhang · Sipeng Zheng · Ziheng Xi · Chaoyi Xu · Haiweng Xu · Haoqi Yuan · Chi Zhang · Yiqing Wang · Yicheng Feng · Zongqing Lu

Preprint posted · CoRL 2026 (accepted) ↗

Robotics17 min text · CondensedOriginal paper ↗
Contents

Overview

The original report credits the BeingBeyond Team collectively; the twelve individual names in this library entry appear in its contributor appendix, reproduced with roles below. This edition covers all 44 pages of arXiv v1, posted 19 January 2026; the abstract page internally dates the report 20 January. Figure and appendix labels follow the PDF, including its missing HTML cover figure.

Complete original Figure 1.
Figure 1: Being-H0.5 at a Glance. We scale human-centric robot learning with Being-H0.5 toward cross-embodiment generalization. We introduce UniHand-2.0, a large-scale corpus exceeding 35,000 hours that spans both cross-Embodiment physical control and general visual-text understanding. Building on this data, we unify human hand motion and diverse robot embodiments with a Unified Action Space, and train all heterogeneous supervision through unified sequence modeling under a single framework. This yields a single foundation model that can perceive, describe, and act within one framework, enabling robust cross-embodiment generalization and real-world deployment across diverse robots and tasks. We empirically deploy a single checkpoint of Being-H0.5 to control PND Adam-U, Franka+Inspire, Unitree G1, BeingBeyond D1, and LeRobot SO-101 to accomplish diverse tasks.

At a glance

Being-H0.5 combines human video, robot demonstrations and vision-language data in one physical action interface. Its flow-based action expert shares lower layers while routing upper layers to specialists; additional gating and asynchronous chunking address noisy context and inference delay.

A jointly adapted checkpoint controls five physical platforms. Simulation specialists score 98.9% on LIBERO and 53.9% on RoboCasa; a checkpoint trained on both scores 97.6% and 53.3%. Transfer to an unseen task–robot combination is qualitative and unquantified, and the masked-motion ablation contains an unresolved table/prose contradiction.

1 Introduction

The central proposal is to use human hand interaction as a common source of manipulation priors while reserving semantically aligned action slots for different robot bodies. This separates shared task structure from hardware-specific commands, although mappings and downstream robot demonstrations remain necessary.

UniHand-2.0 combines roughly 16,000 hours of human video, 14,000 hours of robot data and 5,000 equivalent hours of visual-language data, advertised as 400 million samples and 120 billion tokens. UniCraftor adds synchronized human capture; Mixture of Flow, slot-wise adaptation, reliability gating and asynchronous execution provide the model and deployment components.

The author project page is the report’s reference for resources. The paper describes or promises weights, training and deployment code, simulation scripts and a 1,000-GPU-hour recipe; those promises are distinct from a fully specified, reproduced training run. Claims of the largest or first system are historical author claims.

How can robotic intelligence master a new platform with limited data, much like a human learning a minor language?

2 Related Work

Earlier VLAs use discrete autoregression, diffusion/flow action heads or intermediate plans such as language, points and trajectories. Heterogeneous robot datasets expand hardware coverage but introduce inconsistent control formats and units.

Human-video approaches range from representation learning and latent actions to future-video prediction, geometric tracks and reconstructed hand motion. Being-H0.5 extends Being-H0 with a broader human–robot interface, while building on Open X-Embodiment and other robot corpora. The comparison motivates a representation and data recipe rather than proving every previous VLA is confined to one body.

Complete original Figure 2.
Figure 2: Overview of UniHand 2.0. UniHand 2.0 is our large-scale pre-training recipe for human-centric robot learning, comprising 35K hours of multimodal data from three complementary sources. 1) Human Demonstration with diverse scenes, multitask for motion alignment, and multi-granularity semantics. 2) Robot Manipulation spanning 30+ embodiments with multiple observation views and heterogeneous control signals. 3) Vision–Text Understanding covering general VQA, 2D spatial grounding & affordance, and task planning & reasoning.

3 UniHand-2.0: A Recipe for Large-Scale Human-Centric Learning

The recipe addresses limited data, narrow hardware coverage, scarce dexterous demonstrations and weak language supervision. It counts embodiments with more than ten hours of data and reports 30 robot types, 134 million human samples and over 400 million total samples.

The scale figures use different accounting units. Table 1 totals 13,817.4 robot hours, rounded to 14,000; visual-language data contributes 5,000 equivalent hours rather than recorded interaction time. Figure 4’s caption assigns 25.6B/45.7B/50.2B tokens to human/robot/vision-language data, while the graphic’s legend assigns 45.7B/50.2B/25.6B. The total is about 121.5B, but that source allocation conflict is unresolved. “35,000 hours” is therefore a mixed, rounded aggregate.

Complete original Figure 3.
Figure 3: Comparison of training scale and embodiment diversity. Stacked bars represent hours of training data (left axis); hatched bars represent embodiment counts (right axis). UniHand-2.0 represents the largest and most diverse VLA pre-training recipe to date, totaling 35,000 hours of multimodal data. This includes 16,000 hours of human data, 14,000 hours of robot data across 30 embodiments, and 5,000 equivalent hours of VLM data.
Complete original Figure 4.
Figure 4: Statistics of UniHand-2.0. (Left) Ratio of simulation vs. real-world data. We maintain a balanced ratio with simulation data at 26%, while the widely used Open X-Embodiment (OXE) and AgiBot World datasets account for 3.1% and 3.0%, respectively. (Right) Training sources and scale. Human, robot, and visual-language (“VLM”) data consist of 16K hours (25.6B tokens), 14K hours (45.7B tokens), and 5K equivalent hours (50.2B tokens), respectively. These sources are curated to maintain a comparable scale for balanced pretraining.

3.1 Human Demonstration Data

Public egocentric sources include Ego 4D, EPIC-KITCHENS and Egocentric-10K. HaWoR estimates MANO hand motion and camera poses; Gemini 2.5 supplies per-second instructions and ten-second intent descriptions. Human motion supervises generation, description and continuation.

Processing diversifies language, filters low-confidence or discontinuous motion using detection/DBA signals, removes non-manipulation clips, and mirrors samples to reduce handedness bias. These are estimated labels, not direct measurements of every contact or force. Filter thresholds, error distributions and a contamination audit are not reported.

Table 1: Statistical overview of robot manipulation data in UniHand-2.0. The corpus aggregates demonstrations across 30 distinct embodiments, totaling 14,000 hours of interaction. The dataset is characterized by high diversity in camera views, kinematic structures, and operational environments, categorized by end-effector type—“Grp” (parallel gripper) vs. “Dex” (dexterous hand), and data source—“Real” (physical world) vs. “Sim” (simulation).
Robot Type Camera Views EEF Type Source Hours
Single-Arm Robots
Franka ego, 3rd×\times2, wrist Grp/Dex Real/Sim 2196.4
Google Robot ego, 3rd Grp Real/Sim 1195.2
Kuka-iiwa ego, 3rd Grp Real/Sim 338.2
WidowX 3rd×\times2, wrist Grp Real/Sim 305.2
Xarm7 Robot top, 3rd Grp Real/Sim 594.4
UR5E 3rd, wrist Grp Real/Sim 208.3
Hello Stretch ego, 3rd Grp Real 270.3
Kinova Jaco 3rd×\times3 Grp Real/Sim 194.6
Kinova Gen3 3rd×\times3 Grp Sim 197.4
Sawyer 3rd Grp Sim 196.1
Cobotta 3rd Grp Real 10.6
DLR Sara 3rd Grp Real 12.1
Willow Garage PR2 ego Grp Real 11.9
RMC RM65 3rd Dex Real 41.3
Dual-Arm Robots
RMC Aida L high, wrist×\times2 Grp Real 325.7
Galaxea R1 Lite high, wrist×\times2 Grp Real 630.5
Agilex Split ALOHA high, wrist×\times2 Grp Real/Sim 1099.1
Agilex Cobot Magic high, wrist×\times2 Grp Real 399.2
Piper ego, wrist×\times2 Grp Real/Sim 904.5
Portable Education Arm
BeingBeyond D1 ego Dex Real 100.0
SO101 ego, wrist×\times2 Grp Real 194.4
Half-Humanoid
PND AdamU ego Dex Real 200.0
Agibot-G1 {ego, wrist}×\times2 Grp/Dex Real/Sim 2391.7
Leju kuavo 4 LB ego, wrist×\times2 Grp/Dex Real 1198.2
AlphaBot 2 ego, wrist×\times2 Grp Real 13.8
Airbot MMK2 3rd, wrist×\times2 Grp Real 23.1
Galbot G1 wrist×\times2 Grp Sim 195.8
Tianqing A2 high, wrist×\times2 Grp Real 19.4
Humanoid
Unitree G1edu-u3 ego, wrist×\times2 Grp/Dex Real 135.7
Tiankung top Grp Real 214.3
Total: 30 embodiments 13817.4

3.2 Robot Manipulation Data

Robot data combines public real/simulated corpora with new Adam-U and D1 demonstrations. The pipeline deduplicates and downsamples frames to 30%, adds procedural simulation and inpainting augmentation, and standardizes 30 embodiments.

The text describes simulation as 26% of the total corpus, but Figure 4’s breakdown applies that fraction to the 13,817-hour robot corpus. It should not be read as 26% of all human, robot and language training. Table 1 retains each platform’s hours, views and end-effector type.

3.3 Visual-Text Understanding Data

Auxiliary data covers general image/video question answering, spatial grounding/affordances, and task decomposition. Examples include LLaVA/FineVision, RefCOCO/RoboPoint and ShareRobot/EO1.5M-QA. The purpose is to retain language and scene reasoning during action learning.

The reported 45.7B visual versus 30M text robot tokens imply about 1,523:1, although the prose rounds this to roughly 1,000:1. Adding language supervision is motivated by that imbalance; the paper does not separately measure preservation of each claimed reasoning capability.

4 UniCraftor: A System for Portable, Extensible, and Affordable Data Collection

UniCraftor uses head-mounted RealSense D435 RGB-D, five tabletop AprilTags for PnP camera poses, and a synchronized foot pedal for demonstrator-marked interaction events. Additional views can move while tags remain visible; the system collects over 200 hours across 43 tabletop tasks.

Post-processing inpaints tags with Grounded-SAM2/DiffuEraser, refines HaWoR hands using multiview depth, and generates Qwen 2.5-VL descriptions followed by human verification. Tactile sensing and mobile capture are planned extensions. Tag-based poses and pedal timing are supervision signals with measurement/human error, despite the report’s “ground truth” wording; calibration accuracy, event latency and system cost are not quantified.

Complete original Figure 5.
Figure 5: An overview of our data collection system UniCraftor.

5 Being-H0.5: A Foundational VLA Unifying Cross-Embodiment Control

The model combines a shared multimodal backbone, a physically organized state/action representation and action-specialized transformer computation.

5.1 Model Architecture

Initialized from InternVL3.5, the Mixture-of-Transformers architecture separates understanding and action parameters while sharing attention over a common sequence. Text uses next-token prediction, discrete hand motion uses masked prediction, and continuous control uses rectified flow.

The structure follows BAGEL-style modality specialization. Claimed planning and reasoning functions are not accompanied by a separate reasoning benchmark. Simulation uses a 2B backbone; complete active/total action-expert counts are not specified.

Complete original Figure 6.
Figure 6: Overview of Being-H0.5. Being-H0.5 is a specialized MoT that disentangles multimodal understanding (Und. Expert) and action generation (Act. Expert) while maintaining coupling through shared attention mechanisms. A unified state–action space supports cross-embodiment pre-training by mapping human hand motion and diverse robot controls into semantically aligned slots. Our pre-training leverages UniHand-2.0 by serializing multimodal data into a unified QA-style format, with each modality allocated to the relevant branch. Finally, a Mixture-of-Flow design scales action capacity by combining shared foundation layers with routed specialized experts for embodiment/task-specific dynamics.

5.1.1 Unified State-Action Space

Fixed-length state and action vectors reserve slots for wrist/end-effector poses, joints, grippers, fingers and base commands. Human MANO wrist/finger parameters occupy the corresponding semantic slots; unused dimensions are zero. Cartesian commands use world-frame relative displacements, rotations use axis-angle, and joint positions use absolute radians.

Raw physical scales are retained with outlier filtering instead of statistical normalization. This is a designed interface, not automatic discovery of kinematic equivalence. Slot dimensions/mappings and filters are not fully tabulated; axis-angle also has representation ambiguities near 180° despite avoiding Euler gimbal lock.

5.1.2 Mixture of Flow

Mixture of Flow shares the action expert’s lower transformer layers and routes upper-layer computation to a sparse subset of specialized experts according to state and instruction. Shared primitives can support transfer while selected paths specialize to hardware or tasks.

The report does not give the expert count, routing top-k, balancing loss or complete parameter budget. Sparse activation reduces computation relative to using every expert, but does not itself prove edge-memory fit or conflict-free learning in shared layers. Orin NX feasibility is asserted without a latency/memory table.

5.2 Pre-Training: Human-Centric Robot Learning

Human traces supply varied interaction examples, robot trajectories anchor executable controls, and visual-language tasks maintain semantic context. All three are serialized through one interface, with losses attached only to available supervised segments.

5.2.1 Unified Sequence Modeling

Embodiment maps place raw state/action values into common slots. Each sequence contains tagged vision, text, state and/or action segments, organized as a query and an answer. Training supervises the answer while conditioning on the query. Equations 1–3 and the retained definitions specify these mappings and modality sets.

Equations 1–3
𝐬=Φe​(𝐬(e)),𝐚=Φe​(𝐚(e))\mathbf{s}=\Phi_{e}(\mathbf{s}^{(e)}),\quad\mathbf{a}=\Phi_{e}(\mathbf{a}^{(e)}) (1)
𝒮=[𝐱1,𝐱2,…,𝐱K]\mathcal{S}=[\mathbf{x}_{1},\mathbf{x}_{2},\dots,\mathbf{x}_{K}] (2)
ℳ={vision, text, state, action}\mathcal{M}=\{\text{vision, text, state, action}\} (3)

𝐬∈ℝd\mathbf{s}\in\mathbb{R}^{d}

𝐚∈ℝd\mathbf{a}\in\mathbb{R}^{d}

𝐱k=⟨mk,Ck⟩\mathbf{x}_{k}=\langle m_{k},C_{k}\rangle

mk∈ℳm_{k}\in\mathcal{M}

ℳi⊆ℳ\mathcal{M}_{i}\subseteq\mathcal{M}

[𝒮Q;𝒮A][\mathcal{S}_{Q};\mathcal{S}_{A}]

5.2.2 Human-Centric Multi-Task Objective

Motion generation predicts actions from visual, linguistic and state context. Motion description predicts text from observations and interaction traces; continuation predicts later motion from history. Robot-only and VQA samples instantiate the available modalities.

The joint objective weights text cross-entropy and action losses. This common serialization supports heterogeneous data, but the report does not provide the full mixture schedule or the loss weights needed to reproduce pretraining.

Equations 4–5
ℒ=λtext​ℒtext+λact​ℒact.\mathcal{L}=\lambda_{\text{text}}\mathcal{L}_{\text{text}}+\lambda_{\text{act}}\mathcal{L}_{\text{act}}. (4)
ℒtext=−∑i∈Ωtextlogpθ(yi∣𝒮<i).\mathcal{L}_{\text{text}}=-\sum_{i\in\Omega_{\text{text}}}\log p_{\theta}(y_{i}\mid\mathcal{S}_{<i}). (5)

𝒮A={action}\mathcal{S}_{A}=\{\texttt{action}\}

𝒮A={text}\mathcal{S}_{A}=\{\texttt{text}\}

5.2.3 Hybrid Human Motion Representation

Each human motion chunk has continuous action targets and discrete tokens from a pretrained motion tokenizer. Flow matching regresses the velocity from Gaussian noise toward the target along a linear path; masked-token cross-entropy reconstructs selected codebook entries.

The intent is to combine precise continuous control with a stable discrete motion prior. Tokenizer/codebook details, masking ratio and loss weights are not specified here. The claimed empirical benefit requires caution because Table 8’s higher-is-better scores favor removing the masked objective.

Equations 6–8
ℒact=λ1​ℒFM+λ2​ℒMASK.\mathcal{L}_{\text{act}}=\lambda_{1}\mathcal{L}_{\text{FM}}+\lambda_{2}\mathcal{L}_{\text{MASK}}. (6)
ℒFM=∑i∈ΩFM‖vθ​(𝐱t,t,c)−(𝐚i−𝐱0)‖22.\mathcal{L}_{\text{FM}}=\sum_{i\in\Omega_{\text{FM}}}\left\|v_{\theta}(\mathbf{x}_{t},t,c)-(\mathbf{a}_{i}-\mathbf{x}_{0})\right\|_{2}^{2}. (7)
ℒMASK=−∑i∈ΩMASKlogpθ(zi∣c).\mathcal{L}_{\text{MASK}}=-\sum_{i\in\Omega_{\text{MASK}}}\log p_{\theta}(z_{i}\mid c). (8)

𝐀=[𝐚1,…,𝐚T]∈ℝT×d\mathbf{A}=[\mathbf{a}_{1},\ldots,\mathbf{a}_{T}]\in\mathbb{R}^{T\times d}

𝐚t∈ℝd\mathbf{a}_{t}\in\mathbb{R}^{d}

𝐳∈{1,…,|ℂ|}Tz\mathbf{z}\in\{1,\ldots,|\mathbbm{C}|\}^{T_{z}}

𝐱0∼𝒩⁡(𝟎,𝐈)\mathbf{x}_{0}\sim\mathcal{N}(\mathbf{0},\mathbf{I})

𝐱t=(1−t)​𝐱0+t​𝐚i\mathbf{x}_{t}=(1-t)\mathbf{x}_{0}+t\mathbf{a}_{i}

t∈[0,1]t\in[0,1]

𝐮t​(𝐱t)=𝐚i−𝐱0\mathbf{u}_{t}(\mathbf{x}_{t})=\mathbf{a}_{i}-\mathbf{x}_{0}

ΩMASK⊆{1,…,Tz}\Omega_{\text{MASK}}\subseteq\{1,\ldots,T_{z}\}

Joint Serialization.

Both target segments attend to the same query, but cannot attend to each other, preventing direct copying between continuous and discrete labels. The 3×3 block gate in Equation 9 enforces that separation. Equation 10 gives both target segments the same positional origin after the query; each retains its own relative positions.

Equations 9–10
G=[100110101],for[𝒮Q;𝒮AFM;𝒮AMASK].G\;=\;\begin{bmatrix}\textbf{1}&\textbf{0}&\textbf{0}\\ \textbf{1}&\textbf{1}&\textbf{0}\\ \textbf{1}&\textbf{0}&\textbf{1}\end{bmatrix},\qquad\text{for}\ \ [\,\mathcal{S}_{Q}\,;\,\mathcal{S}_{A}^{\text{FM}}\,;\,\mathcal{S}_{A}^{\text{MASK}}\,]. (9)
PE​(j)={j,j∈𝒮Q,p0+r⁡(j),j∈𝒮AFM,p0+r⁡(j),j∈𝒮AMASK.\text{PE}(j)=\begin{cases}j,&j\in\mathcal{S}_{Q},\\ p_{0}+r(j),&j\in\mathcal{S}_{A}^{\text{FM}},\\ p_{0}+r(j),&j\in\mathcal{S}_{A}^{\text{MASK}}.\end{cases} (10)

[𝒮Q;𝒮AFM;𝒮AMASK][\mathcal{S}_{Q}\,;\,\mathcal{S}_{A}^{\text{FM}}\,;\,\mathcal{S}_{A}^{\text{MASK}}]

p0=maxj∈𝒮Q⁡(j+1)p_{0}=\max_{j\in\mathcal{S}_{Q}}(j+1)

5.3 Post-Training: Towards Cross-Embodiment Adaptation

Post-training must adapt control to new hardware while preserving shared representations. The three mechanisms target slot-specific adaptation, uncertain conditioning features and mismatched inference/control timing.

5.3.1 Embodiment-Specific Adaptation

Embodiment-specific adaptation maintains an adapter bank indexed by semantic action slots. Only active-slot adapters update for a given robot; robots with shared components can share adapter subsets. Inactive-slot updates are explicitly zero in Equation 12.

This localizes some specialization, but shared slots and backbone parameters can still receive competing gradients. The equations describe parameter masking, not a guarantee of collision-free actions or universally conflict-free transfer.

Equations 11–12
minθ⁡𝔼e​𝔼(𝐚0,H)∼𝒟e​𝔼t​‖vθ​(𝐚t,H,e)−v∗​(𝐚t,𝐚0)‖22.\min_{\theta}\ \mathbb{E}_{e}\,\mathbb{E}_{(\mathbf{a}_{0},H)\sim\mathcal{D}_{e}}\,\mathbb{E}_{t}\,\big\|v_{\theta}(\mathbf{a}_{t};H,e)-v^{*}(\mathbf{a}_{t},\mathbf{a}_{0})\big\|_{2}^{2}. (11)
𝐖ESA(e)≜{𝐖ESA​[k]:k∈ℐe},Δ​𝐖ESA​[k]=𝟎∀k∉ℐe.\mathbf{W}_{\text{ESA}}^{(e)}\triangleq\{\mathbf{W}_{\text{ESA}}[k]:k\in\mathcal{I}_{e}\},\qquad\Delta\mathbf{W}_{\text{ESA}}[k]=\mathbf{0}\ \ \forall k\notin\mathcal{I}_{e}. (12)

p⁡(𝐚∣H,e)p(\mathbf{a}\mid H,e)

𝒮={1,…,K}\mathcal{S}=\{1,\ldots,K\}

ℐe⊆𝒮\mathcal{I}_{e}\subseteq\mathcal{S}

𝐖ESA∈ℝK×dout×din\mathbf{W}_{\text{ESA}}\in\mathbb{R}^{K\times d_{\text{out}}\times d_{\text{in}}}

5.3.2 Manifold-Preserving Gating

MPG modifies state/action suffix features while leaving the visual-language prefix unchanged. It compares projected, layer-normalized features with a pooled noise-free action anchor using random-projection sliced-Wasserstein discrepancy. An exponential gate shrinks as discrepancy rises; training stops gradients through that gate.

The gate scales the feature-dependent residual before a linear projection’s bias is added. A low gate therefore leaves the original features plus an ungated learned offset, rather than discarding unreliable observations. Training anchors use target actions; inference anchors use a previous prediction.

The stated variance comparison is a heuristic: removing bias from the gated term does not universally reduce its squared norm because vector cancellation is possible. The Jacobian approximation ignores gate derivatives and retains an identity path. These formulas do not prove robustness or constrain outputs to a valid physical manifold. Gate temperature, projection count and residual strength are unspecified.

Complete original Figure 7.
Figure 7: MPG and UAC Overview. Left (MPG): We compare observation embeddings with a reference action embedding (Train: ground truth; Inference: previous iterate) in the Sliced-Wasserstein Distance (SWD) space to obtain a discrepancy-guided gate gg. The gate scales a feature-conditioned residual while an ungated learned prior offset provides a stable fallback, producing enhanced context features H~\tilde{H} for the action expert. Right (UAC): Based on embodiment-specific dynamic delay dd, each predicted action chunk is split into a committed prefix 𝐀<d\mathbf{A}_{<d} (already queued/executing) and a predicted postfix 𝐀≥d\mathbf{A}_{\geq d}. A dual-thread buffer enables asynchronous inference/execution across robots with heterogeneous latency budgets.
Equations 13–16
H~=H+λ⋅𝒫MPG​(g⋅ℰobs​(H))=H+λ​g​𝐖MPG​ℰobs​(H)+λ​𝐛MPG,\tilde{H}=H+\lambda\cdot\mathcal{P}_{\text{MPG}}\!\big(g\cdot\mathcal{E}_{\text{obs}}(H)\big)=H+\lambda g\,\mathbf{W}_{\text{MPG}}\mathcal{E}_{\text{obs}}(H)+\lambda\mathbf{b}_{\text{MPG}}, (13)
D⁡(μH^,μZ^)≈1M​∑m=1M‖sort​(θm⊤​H^)−sort​(θm⊤​Z^)‖22,D(\mu_{\hat{H}},\mu_{\hat{Z}})\approx\frac{1}{M}\sum_{m=1}^{M}\left\|\text{sort}(\theta_{m}^{\top}\hat{H})-\text{sort}(\theta_{m}^{\top}\hat{Z})\right\|_{2}^{2}, (14)
g=exp(−D/τ)∈(0,1].g=\exp(-D/\tau)\in(0,1]. (15)
∂H~∂H≈𝐈+λ​g​𝐖MPG​∂ℰobs​(H)∂H,\frac{\partial\tilde{H}}{\partial H}\approx\mathbf{I}+\lambda g\,\mathbf{W}_{\text{MPG}}\,\frac{\partial\mathcal{E}_{\text{obs}}(H)}{\partial H}, (16)

𝐚t−Δ​t=𝐚t+Δ​t​vθ​(𝐚t,H)\mathbf{a}_{t-\Delta t}=\mathbf{a}_{t}+\Delta t\,v_{\theta}(\mathbf{a}_{t};H)

H=H⋆+ϵH=H^{\star}+\epsilon

‖∂vθ/∂H‖2​Var​(ϵ)\big\|\partial v_{\theta}/\partial H\big\|^{2}\,\mathrm{Var}(\epsilon)

𝒫MPG​(⋅)=𝐖MPG​(⋅)+𝐛MPG\mathcal{P}_{\text{MPG}}(\cdot)=\mathbf{W}_{\text{MPG}}(\cdot)+\mathbf{b}_{\text{MPG}}

H~=H+λ​g​ℛ​(H)\tilde{H}=H+\lambda g\mathcal{R}(H)

‖𝐖​ℰobs​(H)+𝐛‖22\|\mathbf{W}\mathcal{E}_{\text{obs}}(H)+\mathbf{b}\|_{2}^{2}

‖𝐖​ℰobs​(H)‖22\|\mathbf{W}\mathcal{E}_{\text{obs}}(H)\|_{2}^{2}

Z¯=MeanPool​(Znf)\bar{Z}=\text{MeanPool}(Z^{\text{nf}})

H^=LN​(ℰobs​(H))\hat{H}=\text{LN}(\mathcal{E}_{\text{obs}}(H))

Z^=LN​(ℰact​(Z¯))\hat{Z}=\text{LN}(\mathcal{E}_{\text{act}}(\bar{Z}))

gsg=stopgrad​(g)g^{\text{sg}}=\text{stopgrad}(g)

5.3.3 Universal Async Chunking

UAC extends training-time real-time chunking by sampling delay according to embodiment control period and expected inference latency. Already committed prefix actions are clean; future postfix actions are noisy and receive the flow loss.

Equations 17–19 define the delay distribution, per-token time and postfix-only objective. Exact platform delay distributions and maxima are not provided. UAC learns continuation under timing constraints; it does not remove the need to budget compute and buffer capacity.

Equations 17–19
d∼π(e)​(d),d∈{0,1,…,dmax(e)−1},d\sim\pi^{(e)}(d),\quad d\in\{0,1,\ldots,d_{\max}^{(e)}-1\}, (17)
ti=𝟙[i<d]+𝟙[i≥d]⋅tbase,tbase∼p(t).t_{i}=\mathbbm{1}[i<d]+\mathbbm{1}[i\geq d]\cdot t_{\text{base}},\quad t_{\text{base}}\sim p(t). (18)
ℒUAC=∑i≥d‖v^i−vi∗‖22.\mathcal{L}_{\text{UAC}}=\sum_{i\geq d}\left\|\hat{v}_{i}-v_{i}^{*}\right\|_{2}^{2}. (19)

⌈L(e)/Δ​t(e)⌉\lceil L^{(e)}/\Delta t^{(e)}\rceil

6 Infrastructure: Real-Time Cross-Embodiment Deployment

Deployment combines cached visual-language prefix computation, optional MPG refinement and a two-thread action buffer. These are complementary measures for throughput, feature uncertainty and execution timing.

6.1 Manifold-Preserving Refinement

Euler integration transports Gaussian actions through the learned flow, with fewer than ten steps in the reported implementation. Prefix features are cached while action-token embeddings change. The notation here uses H for the broader context, whereas the MPG subsection uses H specifically for its suffix; the intended modified component remains the suffix.

Equations 20
𝐚(k+1)=𝐚(k)+Δ​t⋅vθ​(𝐚(k),t(k)∣H),Δ​t=1/K,t(k)=k/K,\mathbf{a}^{(k+1)}=\mathbf{a}^{(k)}+\Delta t\cdot v_{\theta}(\mathbf{a}^{(k)},t^{(k)}\mid H),\qquad\Delta t=1/K,\ \ t^{(k)}=k/K, (20)

𝐚(0)∼𝒩⁡(0,𝐈)\mathbf{a}^{(0)}\sim\mathcal{N}(0,\mathbf{I})

Two-Stage Refinement with MPG.

First generate an ungated baseline action chunk. Then perform one to three refinement rounds, each using the preceding prediction’s pooled zero-noise embedding as the MPG anchor throughout a fresh flow-denoising pass. A bad initial prediction can therefore also provide a bad anchor; iterative refinement is not an independently verified ground-truth reference.

Equations 21
Z¯(n−1)=MeanPool​(Enc​(𝐚pred(n−1),σ=0)),𝐚pred(n)=FlowDenoise​(H,Z¯(n−1)),\bar{Z}^{(n-1)}=\text{MeanPool}\!\left(\text{Enc}(\mathbf{a}^{(n-1)}_{\text{pred}},\,\sigma=0)\right),\qquad\mathbf{a}^{(n)}_{\text{pred}}=\text{FlowDenoise}\!\left(H;\bar{Z}^{(n-1)}\right), (21)

Nref∈{1,2,3}N_{\text{ref}}\!\in\!\{1,2,3\}

Why MPG Helps in the Low-Step Regime.

The authors report saturation after two or three refinement rounds at fewer than ten Euler steps. No dedicated step-count/latency curve substantiates that statement here. Each refinement adds another flow pass, so fewer integration steps alone does not establish lower total latency.

Equation

H~≈H+λ​𝐛MPG\tilde{H}\approx H+\lambda\mathbf{b}_{\text{MPG}}

6.2 Universal Async Chunking Protocol

The execution protocol commits the portion of the existing buffer expected to run while inference is busy, keeps it fixed during denoising, and writes only the newly predicted suffix back.

Latency Commitment.

The committed prefix must cover the ceiling of inference time divided by the robot’s control period, plus a timing margin. Underestimating it permits the model to revise actions that already execute. Overestimation protects that commitment only within available chunk/buffer capacity and sacrifices responsiveness; it is not a general physical-safety guarantee.

Equation 22
d≥⌈tinference/tcontrol(e)⌉+ϵsafety,d\ \geq\ \left\lceil t_{\text{inference}}/t_{\text{control}}^{(e)}\right\rceil+\epsilon_{\text{safety}}, (22)

Hard Prefix Locking (denoising-time constraint).

At every Euler step, overwrite prefix positions with their committed buffer values. This enforces agreement with actions that cannot still be changed, while allowing the suffix to evolve.

Equation 23
𝐚i(k)←{ℬi,i<d,𝐚i(k),i≥d,∀k∈{0,…,K},\mathbf{a}^{(k)}_{i}\leftarrow\begin{cases}\mathcal{B}_{i},&i<d,\\ \mathbf{a}^{(k)}_{i},&i\geq d,\end{cases}\qquad\forall k\in\{0,\ldots,K\}, (23)

Hard Stitching (buffer update rule).

After inference, discard the generated prefix and stitch only the suffix onto committed actions. Honoring the timing bound prevents conflicting buffer updates; smoothness at the boundary still depends on the learned continuation and low-level controller.

Equation 24
𝐚exec(n)=𝐚<d(n−1)⊕𝐚≥d(n),\mathbf{a}_{\text{exec}}^{(n)}=\mathbf{a}_{<d}^{(n-1)}\ \oplus\ \mathbf{a}_{\geq d}^{(n)}, (24)

6.3 Dual-Thread Deployment Architecture

A fixed-frequency consumer sends buffered actions to the robot. An asynchronous producer reads observations, generates a chunk and writes its postfix at the committed offset. The ring buffer is at least twice the chunk length, with a short mutex-protected read/write region.

Underflow holds the last action or invokes a robot-specific fallback. The fallback is not fully defined, and holding a command is not automatically safe under every control mode or contact condition.

Benefits.

The same protocol is described for 10-Hz manipulators and 50-Hz humanoids, using cloud or Orin NX computation. Per-platform delay, buffer size and margin must change. The manuscript gives no measured worst-case latency, jitter, underflow frequency or power/memory results, so its real-time claim rests mainly on task demonstrations.

7 Experiment

Experiments compare physical robot policies, simulation specialists/generalists and component ablations. Figure 8 preserves the five hardware and camera configurations.

Complete original Figure 8.
Figure 8: Real-robot embodiments. We evaluate on five robotic platforms spanning upper-body humanoids, single-arm dexterous manipulation, and lightweight parallel grippers.

7.1 Comparison across Real Robots

The physical evaluation spans ten named tasks across spatial, long-horizon, bimanual and generalization categories. Clear Table runs on two robots, yielding eleven task–embodiment rows in Table 3. A single jointly post-trained checkpoint is compared with independently adapted specialists.

7.1.1 Experimental Setup

Hardware comprises Adam-U, G1 with LinkerBot O6, Franka FR3 with Inspire Hand, BeingBeyond D1 and SO-101. Each task receives 30–60 minutes of demonstrations. Tables 2–3 preserve hardware degrees of freedom, camera layouts, exact instructions and assignments.

Scratch variants omit UniHand-2.0 pretraining while retaining the architecture and post-training pipeline. The π0.5 baseline uses the same per-embodiment data and is evaluated only as a specialist in this implementation. The reported baseline restriction should not be generalized to every possible π0.5 adaptation.

Table 2: Hardware specifications of heterogeneous real-robot embodiments.
Embodiment Structure Vision
Group Name DoF Joints Hand Camera Configuration
Upper-body Humanoid PND Adam-U 31 Bimanual+Head+Waist Dexterous (6DoF) ZED Mini Movable (ego), dual-camera
Unitree G1 + LinkerBot O6 26 Bimanual Dexterous (6DoF) D435 Fixed (ego)
Single arm + Dexterous hand FR3 + Inspire Hand 13 Single-arm Dexterous (6DoF) 2×\timesD435 Fixed (3rd-person)
BeingBeyond D1 14 Single-arm Dexterous (6DoF) D435 Movable (ego)
Single arm + Gripper LeRobot SO-101 6 Single-arm Gripper D435 Fixed (3rd-person)
Table 3: Real-robot task suite and instructions. Tasks are categorized by their requisite manipulation capabilities.
Category Task Embodiment Instruction
Spatial Arrange Flowers Adam-U “Arrange the flowers neatly into the vase.”
Water Plant FR3+Inspire “Use your hand to hold the sprinkling can and water the plants”
Stack Bowls FR3+Inspire “Hold the edge of the bowls and stack them on the plate one by one.”
Stack Blocks D1 “Stack the blocks following the color order: <ORDER>.”
Long-horizon Drawer: Open →\rightarrow Place →\rightarrow Close FR3+Inspire “Open the drawer, put <OBJECTS> inside, then close the drawer.”
Package: Flip →\rightarrow Scan G1+O6 “Flip the package then scan the barcode with the scanner.”
Bimanual Hand-over →\rightarrow Box Adam-U “Use both hands to pick and pass <OBJECTS> into the box.”
Pack two products →\rightarrow Close lid G1+O6 “Use both hands to put <OBJECTS> into the box, then close the lid.”
Generalization Wipe Whiteboard FR3+Inspire “Use the wiper to clean any marked area of the whiteboard.”
Clear Table Adam-U “Clear the table: put all objects into the box.”
Clear Table SO-101 “Clear the table: put all objects into the plate.”

Evaluation Protocol.

A common black-box server hides policy identity from the human evaluator, randomizes policies and predefined scene layouts, and records binary success/timeout. The stated default is 20 trials per policy per layout.

The number of layouts, exact category aggregation, success tolerances, timeout and confidence intervals are not fully specified. Blinding reduces model-identity bias but does not by itself guarantee statistical validity or exclude all reset/scoring confounds.

Complete original Figure 9.
Figure 9: Example rollouts of representative real-robot task suites. (Only show one task for each embodiment. The full rollouts of all tasks are shown in Appendix B).

7.1.2 Result Analysis

Figure 10 gives category success rates. The specialist/generalist scores are 75/70 spatial, 60/60 long-horizon, 55/45 bimanual and 75/80 generalization. These support useful joint training, with tradeoffs rather than uniform specialist dominance.

Complete original Figure 10.
Figure 10: Real-robot success rates (%) on four task suites. We compare Being-H0.5 specialist/generalist, scratch ablations (both specialist and generalist), and π0.5\pi_{0.5}.

Specialist is strongest, but generalist stays surprisingly close.

The generalist trails by 5 points spatial and 10 bimanual, ties long-horizon and exceeds the specialist by 5 on generalization. Shared skills are a plausible explanation, but the study does not separately identify which overlap or nuisance factor caused each gain.

Clear margin over π 0.5 , especially on long-horizon and bimanual tasks.

π0.5 scores 55/45/40/65 across the four categories. The generalist’s gains are 15/15/5/15 percentage points, so its largest gap is not uniquely bimanual. Without uncertainty estimates these are observed differences, not established statistical significance.

UniHand-2.0 pretraining is crucial for a strong generalist.

Without UniHand pretraining, specialist scores are 50/35/25/60 and generalist scores 35/25/30/50. Pretraining improves all corresponding categories. The scratch generalist actually exceeds the scratch specialist on bimanual tasks, so its degradation is not uniform. This ablation removes the combined human/robot/language recipe, not human data alone.

Embodiment-Level Zero-Shot Transfer.

Adam-U qualitatively transfers package flipping/scanning, drawer interaction and stacking tasks demonstrated on other bodies, without Adam-U demonstrations for those task pairs. Adam-U itself is already represented in training, so this is an unseen task–embodiment pairing, not an entirely unseen robot.

The authors describe nonzero completion but provide no counts, rate or uncertainty, and acknowledge lower precision. These examples suggest compositional transfer; they do not quantify reliable zero-shot deployment or establish a priority claim.

7.2 Comparison on Simulation Benchmarks

Simulation uses 224×224 RGB, language and a 2B backbone. Specialists train separately on LIBERO or RoboCasa; the generalist trains jointly for twice the steps to maintain comparable updates per benchmark. Joint performance is therefore not obtained at the same total training budget.

Table 4: Success rates (%) on the LIBERO benchmark. Being-H0.5 is evaluated against a wide range of state-of-the-art VLA models. We report the mean success rate over 50 evaluation episodes per task. Being-H0.5 achieves leading performance on the overall average, with a significant advantage in LIBERO-Long, which requires complex multi-step reasoning. Best results are bolded, and second-best are underlined.
Method L-Spatial L-Object L-Goal L-Long Average
Diffusion Policy [20] 78.5 87.5 73.5 64.8 76.1
OpenVLA [6] 84.7 88.4 79.2 53.7 76.5
SpatialVLA [28] 88.2 89.9 78.6 55.5 78.1
CoT-VLA [132] 87.5 91.6 87.6 69.0 83.9
π0\pi_{0}-Fast [133] 96.4 96.8 88.6 60.2 85.5
GR00T-N1 [9] 94.4 97.6 93.0 90.6 93.9
π0\pi_{0} [8] 98.0 96.8 94.4 88.4 94.4
F1 [134] 98.2 97.8 95.4 91.3 95.7
InternVLA-M1 [54] 98.0 99.0 93.8 92.6 95.9
Discrete Diffusion VLA [31] 97.2 98.6 97.4 92.0 96.3
π0.5\pi_{0.5} [34] 98.8 98.2 98.0 92.4 96.9
OpenVLA-OFT [91] 97.6 98.4 97.9 94.5 97.1
X-VLA [11] 98.2 98.6 97.8 97.6 98.1
EO1 [94] 99.7 99.8 99.2 94.8 98.2
Being-H0.5(generalist) 97.0 98.2 99.0 96.2 97.6
Being-H0.5(specialist) 99.2 99.6 99.4 97.4 98.9

7.2.1 Results on LIBERO

LIBERO assesses spatial, object, goal and longer task suites under the benchmark’s demonstration and evaluation setup.

Experimental setup.

Inputs include wrist and third-person views. Action chunks have length 8; packed sequences hold 7,680 tokens per GPU with effective batch 128. Specialists train 45,000 steps on four A800 GPUs; generalist training roughly doubles that. Evaluation uses 500 trials per suite, corresponding to 50 per task.

Results and analysis.

The specialist averages 98.9% versus 97.6% for the generalist. Long-suite scores are 97.4% and 96.2%. The specialist leads the listed overall averages but does not beat every row on every suite: X-VLA scores 97.6% on Long, and EO1 is higher on Spatial/Object. Broad state-of-the-art claims are limited to this comparison and date.

Table 5: Success rates (%) on the RoboCasa 24-task benchmark. Being-H0.5 is evaluated across three task categories, averaged over 50 trials per task. Despite using only 224×\times224 RGB inputs, Being-H0.5 outperforms state-of-the-art 3D-based methods and existing VLAs on the overall average. Best results are bolded, and second-best are underlined.
Modality Method Pick & Place Doors/Drawers Others Total Avg.
3D (Multi-modal)
3D 3DA [138] 0.0 2.3 13.1 5.5
3D DP3 [139] 1.5 41.7 32.0 22.8
3D GWM [140] 14.8 54.3 49.8 39.3
RGB-only (Standard)
RGB (256×\times256) BC [136] 4.3 47.0 42.2 28.9
RGB (256×\times256) GR00T-N1 [9] 18.6 50.2 39.1 36.0
RGB (256×\times256) π0.5\pi_{0.5} [34] 21.5 57.8 44.9 41.4
RGB (256×\times256) π0\pi_{0} [8] 14.0 53.1 58.5 42.4
RGB (224×\times224) Being-H0.5(generalist) 40 73 52 53.3
RGB (224×\times224) Being-H0.5(specialist) 36 71.7 57.6 53.9

7.2.2 Results on RoboCasa

RoboCasa tests 24 kitchen tasks using the Human-50 setting and held-out scene/object appearances.

Experimental setup.

Each task provides 50 demonstrations and receives 50 evaluation trials across five held-out scenes. Observations are 224×224 RGB with natural-language commands, without depth or point clouds. Specialist and joint training budgets follow the LIBERO schedule.

Results and analysis.

Overall success is 53.9% specialist and 53.3% generalist. Pick/place scores 36/40, doors/drawers 71.7/73 and other tasks 57.6/52. The generalist improves two categories and loses ground on the third; nearly half of all trials still fail. Baseline architecture, input modality and pretraining differ, so the comparison does not isolate human-data causality.

Takeaway.

One jointly trained checkpoint covers both simulation benchmarks with a modest aggregate tradeoff after doubled training. The specialist is a comparison point, not a mathematical upper bound: the generalist is better on some categories. These simulation checkpoints are distinct from the physical-robot generalist evaluation.

Table 6: Impact of human-centric learning in single-task 5-shot adaptation. We investigate this by selectively freezing key architectural components. Native VLM: Initialized from the original large multimodal model. Human-Centric: Initialized with our human-centric pretrained weights. The action expert remains trainable in all settings. Δ\Delta denotes the absolute performance gain.
Model Configuration Single-Task LIBERO 5-Shot Success Rate (%)
Frozen Components Initialization L-Spatial L-Object L-Goal L-Long Avg.
Und + Proj + ViT Native VLM 67.8 76.8 57.2 29.8 57.9
Human-Centric 78.6 78.9 72.6 46.2 69.0
Gain (Δ\Delta) +10.8 +2.1 +15.4 +16.4 +11.1
Und + ViT Native VLM 70.6 50.0 67.0 17.4 51.3
Human-Centric 85.0 84.2 80.0 59.0 77.1
Gain (Δ\Delta) +14.4 +34.4 +13.0 +41.6 +25.8
Und Only Native VLM 78.4 84.8 62.0 49.2 68.6
Human-Centric 83.2 79.6 74.0 63.6 75.1
Gain (Δ\Delta) +4.8 -5.2 +12.0 +14.4 +6.5
Proj + ViT Native VLM 87.2 86.6 80.8 58.0 78.2
Human-Centric 86.2 86.2 84.6 66.8 81.0
Gain (Δ\Delta) -1.0 -0.4 +3.8 +8.8 +2.8
None (Full FT) Native VLM 84.2 83.6 75.0 65.8 77.2
Human-Centric 87.2 91.4 81.6 66.8 81.8
Gain (Δ\Delta) +3.0 +7.8 +6.6 +1.0 +4.6
Table 7: Impact ablation of human-centric learning in multi-task 5-shot adaptation. In contrast to the single-task setup, the model is jointly trained on data from all four task suites and subsequently evaluated on each individually. Native VLM: Initialized from the original large multimodal model. Human-Centric: Initialized with our human-centric pretrained weights. This multi-task protocol serves to rigorously assess the generalization capabilities and robustness of VLAs under diverse cross-task scenarios.
Model Configuration Multi-task-suite LIBERO 5-Shot Success Rate (%)
Frozen Components Initialization L-Spatial L-Object L-Goal L-Long Avg.
Und + Proj + ViT Native VLM 61.6 77.0 68.0 36.0 60.7
Human-Centric 80.8 81.8 74.4 52.4 72.4
Gain (Δ\Delta) +19.2 +4.8 +6.4 +16.4 +11.7
Und + ViT Native VLM 78.4 84.8 71.4 38.6 68.3
Human-Centric 83.6 82.6 80.8 59.2 76.6
Gain (Δ\Delta) +5.2 -2.2 +9.4 +20.6 +8.3
Und Only Native VLM 75.8 84.2 71.2 59.8 72.8
Human-Centric 85.2 90.6 84.2 68.4 82.1
Gain (Δ\Delta) +9.4 +6.4 +13.0 +8.6 +9.3
Proj + ViT Native VLM 87.6 91.0 82.2 62.6 80.9
Human-Centric 89.6 88.8 89.6 65.8 83.5
Gain (Δ\Delta) +2.0 -2.2 +7.4 +3.2 +2.6
None (Full FT) Native VLM 87.8 94.6 82.6 71.4 84.1
Human-Centric 88.8 90.2 85.4 76.0 85.1
Gain (Δ\Delta) +1.0 -4.4 +2.8 +4.6 +1.0
Complete original Figure 11.
Figure 11: Ablation study on the number of frozen layers in the action expert. Results compare the performance of the model after human-centric pretraining (“pt”) and without it (“w/o pt”). “MoF” denotes our proposed Mixture of Flow architecture. All experiments freeze all parameters of the MLLM including the visual encoder and projector. Note that each task was evaluated with 20 rollouts due to time constraints.

7.3 Ablation Study

Ablations examine pretrained weights, trainable components, masked human-motion supervision and runtime consistency. They provide different kinds of evidence and should not be treated as one controlled test of the complete causal story.

7.3.1 How Do Human-Centric Learning Benefit Downstream Adaptation?

LIBERO five-shot adaptation uses 10,000 training steps. Separate-suite and joint-suite runs compare native-VLM initialization with the combined UniHand pretrained weights while varying frozen components; the action expert remains trainable in Tables 6–7. The “single-task” table caption refers to suite-specific training in the protocol.

Complete original Figure 12.
Figure 12: Ablation Study on the Number of Frozen Layers in the Action Expert. Results compare the performance of the model after human-centric pretraining (“pt”) and without it (“w/o pt”). “MoF” denotes our proposed Mixture of Flow architecture. All experiments freeze all parameters of the MLLM, including the visual encoder and projector. Note that each task was evaluated with 20 rollouts due to time constraints.

Impact of Backbone Pretraining.

Freezing understanding plus vision while training the projector/action expert yields the largest separate-suite average gain:51.3→77.1, or 25.8 points; Long improves 17.4→59.0, or 41.6 points. Full tuning narrows the average gain to 4.6 points separate-suite and 1.0 joint-suite, with negative transfer on some Object cells.

Full tuning has the highest listed pretrained averages, 81.8 and 85.1, so freezing the backbone is not demonstrated to be globally optimal. Table 6’s Object gain for the frozen-understanding/vision row is printed+34.4 although 84.2−50.0=34.2; all original values remain visible. The prose’s reference to 50 demonstrations also conflicts with its five-shot framing.

Sensitivity of Action Expert Plasticity.

Figures 11–12 freeze progressively more action-expert layers while keeping the multimodal backbone fixed. Early-layer freezing costs little in several suites; extensive freezing sharply degrades most tasks. The Object curve remains comparatively higher than others when the plain pretrained expert is nearly/all frozen, so “below 20% everywhere” overstates the plot.

Each task receives only 20 rollouts. Figure 11’s legend says “pt + MoE” while its caption calls the component MoF; this original notation is retained. Figure 12 reports relative performance and trainable-parameter ratios, not a separate demonstration of new-task transfer.

7.3.2 How Does Masked Motion Token Prediction Benefit Capturing Behavior Priors?

MWDS is mean cosine similarity between predicted and target wrist displacement from start to finish, evaluated on held-out lab and wild human videos. It tests net direction, not complete trajectory accuracy or task completion; treatment of zero displacement and split sizes is unspecified.

Source inconsistency: Table 8 marks higher as better, but Hybrid scores 0.33/0.20 while removing the masked objective scores 0.35/0.28. Both HTML and PDF show these values. The surrounding claim that removing masking lowers MWDS is unsupported by the displayed table; no values or labels are silently reversed.

Table 8: Ablation on masked motion token prediction. MWDS ↑\uparrow on a held-out human demonstration test set with Lab (lab-curated) and Wild (in-the-wild) splits.
Method Lab ↑\uparrow Wild ↑\uparrow
Hybrid (Ours) 0.33 0.20
w/o ℒmask\mathcal{L}_{\textsc{mask}} 0.35 0.28
Complete original Figure 13.
Figure 13: Ablation of MPG+UAC on real robots. Removing MPG+UAC hurts long-horizon and bimanual categories the most, where execution delay and unreliable context amplify compounding errors.
Equation

cos⁡(Δ​𝐩^wrist,Δ​𝐩wrist)\cos\!\left(\Delta\hat{\mathbf{p}}_{\text{wrist}},\,\Delta\mathbf{p}_{\text{wrist}}\right)

7.3.3 Real-Time Efficiency: MPG and UAC

Figure 13 includes separate removal of MPG, UAC and both. Against the full generalist’s 70/60/45/80 scores, removing both gives 50/35/20/55. These are category task rates, not measurements of inference speed or physical-safety guarantees.

Long-horizon drops sharply without UAC-style async consistency.

Long-horizon success falls 60→55 without MPG, 60→40 without UAC and 60→35 without both. This supports a larger role for timing consistency in the tested multi-stage tasks, while the proposed error mechanisms are observational explanations.

Bimanual coordination is brittle without stability mechanisms.

Bimanual success falls 45→40 without MPG and 45→20 without UAC or both. Timing consistency is particularly important here; the aggregate rates alone do not isolate oscillation or camera motion as the cause.

Spatial/generalization are affected but less catastrophically.

Spatial success falls 70→55 without MPG, 70→65 without UAC and 70→50 without both. Generalization falls 80→60 without MPG and 80→55 without both, but remains 80 without UAC. Thus degradation is not universal, and the combined 25-point generalization drop equals the long-horizon/bimanual drops.

8 Conclusion

The report presents a substantial shared-interface and data-engineering approach to human–robot learning, with useful physical multi-platform behavior and strong listed simulation results. Its evidence supports adaptation of a pretrained model to multiple bodies, rather than reliable unrestricted operation on unseen hardware.

Reproduction and interpretation are limited by missing pretraining/routing/MPG parameters, unquantified zero-shot completion, incomplete real-world trial accounting, absent runtime measurements and the masked-motion table contradiction. The combined-data ablation does not isolate the human component, and synthetic labels/scene overlap require further audit.

Appendix A Contributions

Core contributors and model training: Hao Luo, Ye Wang, Wanpeng Zhang and Sipeng Zheng. Infrastructure: Wanpeng Zhang (lead), Ye Wang, Chaoyi Xu, Chi Zhang and Haoqi Yuan.

Human data: Hao Luo (lead), Ziheng Xi, Yiqing Wang and Yicheng Feng. Robot data: Ye Wang and Sipeng Zheng (leads), with Haiweng Xu. Experiments: Wanpeng Zhang, Ye Wang, Hao Luo and Sipeng Zheng. Project leads: Sipeng Zheng and Zongqing Lu.

Appendix B Full Rollouts

Figure 14 retains the complete rollout montage from AppendixB, showing every illustrated real-robot task sequence. These are qualitative examples rather than the raw evaluation trial log.

Complete original Figure 14.
Figure 14: Example rollouts of all real-robot task suites.
Concept demonstration

Open image ↗