Literature
Display
← All papers

MolmoAct2: Action Reasoning Models for Real-World Deployment

Haoquan Fang · Jiafei Duan · Donovan Clay · Sam Wang · Shuo Liu · Weikai Huang · Xiang Fan · Wei-Chuan Tsai · Shirui Chen · Yi Ru Wang · Shanli Xing · Jaemin Cho · Jae Sung Park · Ainaz Eftekhar · Peter Sushko · Karen Farley · Angad Wadhwa · Cole Harrison · Winson Han · Ying-Chun Lee · Eli VanderBilt · Rose Hendrix · Suveen Ellawela · Lucas Ngoo · Joyce Chai · Zhongzheng Ren · Ali Farhadi · Dieter Fox · Ranjay Krishna

Preprint posted · CoRL 2026 (accepted) ↗

Robotics23 min text · CondensedOriginal paper ↗
Contents

Overview

The paper releases model checkpoints, training data and implementation code. The original resource links are collected below.

At a glance

MolmoAct2 combines an embodied-reasoning VLM, curated robot data, an open FAST action tokenizer and a continuous flow-matching expert conditioned on every VLM layer. Its Think variant caches depth tokens and regenerates changed regions. Results are strong on the reported benchmarks, but zero-shot deployment is embodiment-specific, hardware scores sometimes include partial credit, and one-second action chunks execute without within-chunk feedback.

1 Introduction

The project addresses three coupled obstacles: insufficient spatial grounding, the latency of explicit intermediate reasoning, and limited reproducibility of robot foundation models. It extends MolmoAct through a stronger backbone, robot data spanning three deployment platforms, transparent action tokenization, per-layer key/value conditioning and adaptive depth reasoning. Releasing these components supports inspection and adaptation; benchmark success does not establish dependable general-purpose deployment.

This edition covers the complete 51-page v2 of 8 May 2026, including all 14 figures, 25 tables, 23 numbered equations, both implementation listings and the full annotation prompt. The displayed author name Zhongzheng Ren follows this exact source version.

Complete original PDF figure S1.F1
Figure 1: Overview of MolmoAct2. MolmoAct2 is a fully open action reasoning model for real-world deployment. From a suite of high-quality robot datasets that we collect, filter, and curate at scale across three platforms spanning the low-to-medium cost range (left), we train MolmoAct2 and its adaptive-depth reasoning variant MolmoAct2-Think, coupled to the VLM backbone through per-layer KV conditioning (center). The resulting models deploy out-of-the-box on bimanual YAM, SO-100/101, and DROID Franka, and adapt to in-the-wild tasks such as cleaning up, washing dishes, wetlab automation, and pouring tea (right).

2 Molmo2-ER

Molmo2 is specialized for spatial and embodied tasks, then rehearsed on general multimodal data. Molmo2-ER reaches a reported 63.8 benchmark mean versus 46.8 for Molmo2. Its strongest result is the overall average; the source’s claim of beating every baseline on nine individual benchmarks overstates the values in Table 3, as detailed in the results notes.

Complete original PDF table S2.T1
Table 1: MolmoAct2 multimodal web data training corpus (combining data from Molmo2, Molmo2-ER, and Tulu-3). Sizes denote the number of samples used in our mixture (we subsample several datasets to balance the mixture). Sampling weights are the marginal proportion of each group within the non-robotics portion of the pretraining mixture.
Original selectable table
#Samples Weight #Samples Weight
Molmo2 Dataset Molmo2-ER Dataset
Image QA 2.4M 0.115 Image Embodied QA 1.3M 0.11
Video QA 2.4M 0.092 Image Pointing 780K 0.11
Image Pointing 1.1M 0.046 Image Detection 100K 0.01
Video Pointing 370K 0.069 Video Embodied QA 703k 0.10
Video Tracking 800K 0.069 Multi-image/ Ego-Exo 700K 0.09
Captions/Long QA 1.2M 0.069 Abstract Reasoning 150K 0.04
Subtotal 8.3M 0.46 Subtotal 3.3M 0.46
Tulu-3 SFT Dataset 980k 0.08 Total 12.5M 1.0

2.1 Training data

The approximately 3.3-million-example embodied corpus combines six capability groups, from geometric image questions to video reasoning and cross-view correspondence. Supervision comes from simulation, annotated scans, templates and some generated reasoning traces. Table 1 reports sampling weights separately from raw sample counts; the rounded category sizes are not a consistent exact accounting of the stated subtotal.

Image embodied QA.

Image QA mixes SAT’s dynamic spatial reasoning, RoboPoint-QA’s general perception, RefSpatial’s referring and chain-of-thought examples, VST-P’s virtual-camera geometry and VSI-590K’s video-derived spatial supervision. The mixture spans real and synthetic data and multiple viewpoints rather than relying on one question template.

Video embodied QA.

SIMS-VSI contributes 203,000 simulator-grounded video questions about distance, direction, counting and order. A 200,000-example RoboVQA subset adds human-annotated long-horizon planning, affordances and prediction. These train perceptual reasoning, not directly measured robotic success.

Pointing and object detection.

Pixel localization uses 700,000 RoboPoint pointing examples, its 100,000 LVIS detection examples and RefSpatial’s 80,000 pointing examples. The normalized coordinate format grounds referring expressions and free-space descriptions in image locations.

Multi-image and ego–exo correspondence.

A 500,000-example SenseNova-SI subset and 200,000 cross-view VST-P examples train correspondence between cameras and between egocentric and external views. These complement the single-image data used elsewhere in the mixture.

Abstract embodied reasoning.

CLEVR contributes 50,000 compositional attribute/relation examples, while 100,000 GRiD-3D examples emphasize directions relative to an object’s own frame. This specifically separates object-relative language from camera-relative directions.

2.2 Specialize-then-rehearse training recipe

The specialize-then-rehearse recipe adapts a released checkpoint rather than repeating all original multimodal training. The second stage aims to recover general capability without discarding the new spatial specialization.

Stage 1: Embodied specialization.

Starting from the 4B Molmo2 mid-training checkpoint, specialization runs for 20,000 steps with 8% Tulu-3 text data. Sequence length is 4,200 and global batch size 64 on sixteen H100 GPUs. The text component is intended to retain language competence.

Stage 2: Joint refinement.

Joint refinement adds 1,500 steps, splitting the non-text budget equally between embodied and original Molmo2 data after testing embodied fractions 0.3, 0.5, 0.7 and 0.9. The resulting overall mixture is 46% embodied, 46% general multimodal and 8% text. Context grows to 16,384 tokens and device batch size falls to one.

3 Data

The robot mixture concentrates on bimanual YAM, community SO-100/101 and DROID Franka demonstrations, with smaller academic datasets and multimodal rehearsal. Collection and filtering improve practical coverage, but the mix remains heavily concentrated on its three intended deployment platforms.

Complete original PDF figure S3.F2
Figure 2: Overview of the MolmoAct2 training data. Our data mixture combines public academic robot datasets, MolmoAct2-BimanualYAM Dataset, MolmoAct2-DROID Dataset, MolmoAct2-SO100/101 Dataset, multimodal web data, and embodied-reasoning data.

3.1 MolmoAct2-BimanualYAM Dataset

The YAM release contains 34,500 teleoperated demonstrations, over 720 hours and more than 28 tasks, collected over two months with varied objects and layouts. Retry and idle-segment protocols are described as quality controls. Figure 3 shows a setup priced below US$6,000 at publication; that is hardware cost, not the cost of data collection or model training. Table 25 compares a selected set of tabletop bimanual datasets.

3.2 MolmoAct2-SO100/101 Dataset

SO data comes from 1,222 public LeRobot datasets and 377 contributors, totaling 38,059 episodes and roughly 184 hours. Filtering checks schema, corruption, evaluation-only collections and eligibility, then retains datasets whose mean TOPReward on their last three sampled episodes exceeds a human-audited reference threshold. This is a dataset-level proxy gate, not human verification of every episode.

Complete original PDF figure S3.F3
Figure 3: MolmoAct2-BimanualYAM Dataset collection setup. Our standardized MolmoAct2-BimanualYAM Dataset collection setup. Every component is readily available for purchase, and the total cost of the entire setup is under $6,000 USD.

3.3 MolmoAct2-DROID Dataset

The DROID subset uses successful episodes, extended instructions and idle-frame filtering from the supplementary annotations. It contains 74,604 episodes and 17,758,044 frames, with non-idle segments at least one second long. Instructions are re-annotated for the training mix, although the dedicated DROID fine-tune deliberately omits the additional annotations for comparison fairness.

3.4 Language annotation pipeline

Qwen3.5-27B sees sampled multi-camera frames and the original task description, then generates an imperative instruction with a randomized target word count. The reported global unique-label count rises from 71,121 to 146,485. More distinct strings are evidence of linguistic diversity, not an independent audit of semantic correctness; the complete annotation prompt is preserved in Appendix D.

3.5 Academic robotics datasets

BC-Z, BridgeData V2, RT-1 and the earlier MolmoAct dataset broaden camera, task and control conventions. They complement the three dominant robot datasets rather than supplying equal training mass.

3.6 Multimodal datasets

The non-robot portion is 46% Molmo2-ER, 46% original Molmo2 multimodal data and 8% Tulu-3 text. This internal composition is distinct from the overall VLA pretraining mixture, which samples 90% robot examples and 10% non-robot examples.

4 MolmoAct2

Training first creates a discrete robot-aware VLM, then attaches a continuous controller, then specializes to the deployment embodiment or task. Separating these stages lets the token interface learn before continuous expert alignment, and avoids requiring one expensive heterogeneous training run to finish every embodiment’s adaptation.

4.1 Pre-training

Discrete pretraining keeps the VLM’s image/text interface and adds state, setup, control and action tokens. It predicts one-second action chunks with the same next-token objective used for language. The continuous expert is introduced only afterward.

4.1.1 MolmoAct2-FAST Tokenizer

MolmoAct2-FAST follows FAST: a frequency transform, coefficient quantization and byte-pair encoding produce a 2,048-token action vocabulary. The contribution is an inspectable data mixture and released tokenizer rather than a different compression principle. Actions use percentile normalization, separate gripper handling and zero-padding to 32 dimensions.

The detailed recipe specifies one million sampled one-second sequences, despite the abstract’s “millions of trajectories” wording. Table 2 assigns 30% each to YAM, SO and DROID, with the remaining roughly 10% split among Fractal, BC-Z and Bridge. MolmoAct data is mentioned in prose but has no separate tokenizer-table row.

Complete original PDF table S4.T2
Table 2: Embodiment mixture for MolmoAct2-FAST Tokenizer training. The tokenizer is trained on one million subsampled action sequences spanning the main robot embodiments used by MolmoAct2, plus smaller sources that broaden control-mode coverage.
Original selectable table
Dataset Mix Robot Action representation
MolmoAct2-BimanualYAM Dataset 30% YAM Absolute joint
MolmoAct2-SO100/101 Dataset 30% SO-100/101 Absolute joint
MolmoAct2-DROID Dataset 30% Franka Absolute joint
Fractal (Brohan et al., 2022) 3.33% Google Robot Delta end-effector
BC-Z (Jang et al., 2022) 3.33% Google Robot Delta end-effector
Bridge (Walke et al., 2023) 3.33% WidowX Delta end-effector

4.1.2 Pre-training recipe

Model.

The backbone retains SigLIP2 visual encoding, the Molmo2 connector and the 4B language model. Intermediate visual features are pooled and projected into language tokens. Appendix A supplies the architecture and the distinction between image and video pooling.

Data.

Pretraining samples 90% robot and 10% non-robot data; each major robot dataset receives 30% of the robot portion. Images use one resized crop, videos at most eight frames at up to 2 FPS, and camera order is randomized per episode. Setup/control descriptors identify the embodiment and action semantics, while geometric/color augmentation discourages dependence on a fixed visual arrangement.

Action and state representation.

Actions span one second at the dataset’s own rate, are normalized using first/99th-percentile statistics and padded to 32 dimensions before tokenization. State scalars are independently discretized into 256 bins. The shared vector width does not imply a shared physical coordinate system across joint-space and end-effector datasets.

Training.

Pretraining uses 200,000 steps, 4,200-token packed sequences and global batch 128 on 64 H100s, totaling about 5,760 GPU-hours. Attention masks separate packed examples. Learning rates are 5×10⁻⁶ for the vision encoder/connector and 10⁻⁵ for the language model; the new token embeddings also learn.

4.2 Post-training

Post-training adds a DiT-style continuous action expert while retaining discrete action supervision. The expert conditions on projected VLM keys and values at every layer. Ground-truth action-token positions are masked out of that context to prevent target leakage.

Complete original PDF figure S4.F4
Figure 4: Overview of MolmoAct2. Image observations, language instructions, and robot states are tokenized and processed by a pre-trained VLA backbone with self-attention layers. Post-training attaches a DiT-style action expert with the same number of layers as the backbone. At each layer, the backbone key and value tensors are projected and reused as the key and value inputs to the action expert’s cross-attention, enabling per-layer KV conditioning from visual-language context to continuous control. The expert is trained with flow matching to denoise a noisy action trajectory into a continuous robot trajectory. During training, the backbone is also supervised with next-token prediction over discrete action tokens, while the target action-token span is masked from the expert so continuous action prediction cannot condition on the ground-truth discrete actions.

4.2.1 Action expert and per-layer KV conditioning

The flow model interpolates Gaussian noise toward the action chunk and regresses the constant target velocity, masking padded coordinates. Its 36 blocks each contain action self-attention, cross-attention to the corresponding VLM layer and a feed-forward branch, with time-dependent normalization and gates. Adapter projections map VLM keys/values into the expert width.

During post-training, detaching this conditioning path prevents the continuous loss from updating the VLM; the language-model loss still updates it. At inference, a learned velocity field is numerically integrated from noise, following the flow-matching formulation.

Equations 1–7
xt=(1−t)​ϵ+t​a,u⋆=a−ϵ.x_{t}=(1-t)\epsilon+ta,\qquad u^{\star}=a-\epsilon. (1)
ℒflow=𝔼a,ϵ,t​[‖m⊙(fθ​(xt,t,c)−u⋆)‖22],\mathcal{L}_{\mathrm{flow}}=\mathbb{E}_{a,\epsilon,t}\left[\left\|m\odot\left(f_{\theta}(x_{t},t,c)-u^{\star}\right)\right\|_{2}^{2}\right], (2)
hℓ′\displaystyle h^{\prime}_{\ell} =hℓ+gℓsa​SA⁡(AdaRMSℓsa⁡(hℓ,t)),\displaystyle=h_{\ell}+g^{\mathrm{sa}}_{\ell}\operatorname{SA}\!\left(\operatorname{AdaRMS}^{\mathrm{sa}}_{\ell}(h_{\ell},t)\right), (3)
h¯ℓ\displaystyle\bar{h}_{\ell} =hℓ′+gℓca​CA⁡(AdaRMSℓca⁡(hℓ′,t),K~ℓ,V~ℓ),\displaystyle=h^{\prime}_{\ell}+g^{\mathrm{ca}}_{\ell}\operatorname{CA}\!\left(\operatorname{AdaRMS}^{\mathrm{ca}}_{\ell}(h^{\prime}_{\ell},t),\tilde{K}_{\ell},\tilde{V}_{\ell}\right), (4)
hℓ+1\displaystyle h_{\ell+1} =h¯ℓ+gℓff​MLP⁡(AdaRMSℓff⁡(h¯ℓ,t)).\displaystyle=\bar{h}_{\ell}+g^{\mathrm{ff}}_{\ell}\operatorname{MLP}\!\left(\operatorname{AdaRMS}^{\mathrm{ff}}_{\ell}(\bar{h}_{\ell},t)\right). (5)
K~ℓ=reshape⁡(PK​Kℓvlm),V~ℓ=reshape⁡(PV​Vℓvlm).\tilde{K}_{\ell}=\operatorname{reshape}\!\left(P_{K}K^{\mathrm{vlm}}_{\ell}\right),\qquad\tilde{V}_{\ell}=\operatorname{reshape}\!\left(P_{V}V^{\mathrm{vlm}}_{\ell}\right). (6)
CA⁡(Qℓ,K~ℓ,V~ℓ)=softmax⁡(Qℓ​K~ℓ⊤dh)​V~ℓ,\operatorname{CA}(Q_{\ell},\tilde{K}_{\ell},\tilde{V}_{\ell})=\operatorname{softmax}\!\left(\frac{Q_{\ell}\tilde{K}_{\ell}^{\top}}{\sqrt{d_{h}}}\right)\tilde{V}_{\ell}, (7)

4.2.2 Post-training recipe

Overview.

The 200,000-step discrete checkpoint supplies the post-training initialization. Robot examples now carry both discrete action targets and continuous trajectories, while non-robot examples retain language-model supervision.

Multiple flow samples.

Four independent noise/time samples are drawn per action chunk and share the same VLM context. They provide multiple denoising targets for one demonstration without recomputing the backbone. Fine-tuning later uses eight samples; post-training stays at four because of GPU memory limits.

Equation 8
ℒflow​(a,c)=1K​∑i=1K‖m⊙(fθ​(xti,ti,c)−(a−ϵi))‖22.\mathcal{L}_{\mathrm{flow}}(a,c)=\frac{1}{K}\sum_{i=1}^{K}\left\|m\odot\left(f_{\theta}(x_{t_{i}},t_{i},c)-(a-\epsilon_{i})\right)\right\|_{2}^{2}. (8)

Padding and packing.

A shared tensor supports up to 30 time steps and 32 action dimensions. Masks exclude padded steps, coordinates and empty chunk rows from learning. Packed action chunks attend only to their corresponding prompt segment; a five-chunk padding cap improves shape stability when possible without discarding longer packed sequences.

Training objectives.

The objective adds language-model and flow losses. Discrete action targets supervise the backbone but are inaccessible to the expert’s conditioning context. Detached VLM keys/values provide knowledge insulation for the continuous objective while leaving ordinary autoregressive learning active.

Equation 9
ℒpost=ℒLM+ℒflow.\mathcal{L}_{\mathrm{post}}=\mathcal{L}_{\mathrm{LM}}+\mathcal{L}_{\mathrm{flow}}. (9)

Training.

Post-training runs 100,000 updates with global batch 128 on 64 H100s, about 2,304 GPU-hours. Robot sequences use 2,100 tokens, while non-robot batches retain 4,200. The new expert learns at 5×10⁻⁵; backbone learning rates remain unchanged.

4.3 Deployment

4.3.1 Embodiment-specific fine-tuning

Shared recipe.

Embodiment fine-tuning is robot-only, uses eight flow samples per chunk and allows continuous-loss gradients back into the VLM. It retains discrete action co-training, normalization, packing and the existing architecture. Added input-token embeddings are frozen in the main checkpoints while other specified parameters adapt; this is separate from the LoRA and expert-only ablations.

Bimanual YAM.

YAM uses fixed top/left/right cameras, annotated instructions, absolute joint targets and 30-step chunks at 30 Hz. Fine-tuning runs 100,000 updates with batch 128 on 64 H100s, about 2,304 GPU-hours.

DROID.

DROID uses an exterior camera followed by the wrist camera, absolute joint targets and 15-step chunks at 15 Hz. Alternate exterior views may be sampled during training. The dedicated run omits additional language annotations and uses 100,000 updates, batch 64 and 32 H100s, about 1,152 GPU-hours.

SO-100/101.

SO-100/101 randomizes camera order because community view layouts are inconsistent. It uses annotated instructions, absolute joint targets and 30-step chunks at 30 Hz, with 100,000 updates, batch 64 and 32 H100s.

LIBERO.

All four LIBERO suites train jointly with front/wrist cameras and relative end-effector actions. Ten-step chunks correspond to 10 Hz. Training runs 50,000 updates, with the best standard-model checkpoint selected at 40,000; Think selects a different checkpoint at 30,000.

Other evaluation fine-tunes.

Smaller downstream runs use the same robot-only structure, fixed metadata camera order and the dataset’s action rate/convention. The generic real-world recipe specifies 50,000 updates on eight H100s with batch 16 and best-checkpoint selection. Appendix C gives the distinct joint RoboEval recipe of 10,000 updates and batch 64.

4.3.2 Inference optimization

Within a chunk, the VLM context is fixed across ten flow steps. Caching context-dependent expert intermediates and fixed positional terms avoids repeated work; CUDA Graphs reduce launch overhead for the fixed-shape integration loop.

5 MolmoAct2-Think

Think predicts a 10×10 grid of depth codes before generating actions. Each cell uses one of 128 learned codes. Temporal reuse avoids regenerating the entire grid when only a small part of the scene changes, but depth generation still adds substantial latency relative to the action-only policy.

Complete original PDF figure S5.F5
Figure 5: Overview of MolmoAct2-Think. MolmoAct2-Think augments the MolmoAct2 action-generation pipeline with adaptive depth-token reasoning, reusing cached depth codes for static regions and regenerating depth codes for changed regions before conditioning the action expert.

5.1 Adaptive depth perception data

Depth Anything V2 supplies monocular pseudo-depth, which the earlier MolmoAct VQ-VAE compresses from 320×320 pixels to 100 codes. RGB patches use cosine similarity with the previous frame; values below 0.996 trigger updates. Training targets the carried-forward depth buffer rather than independently refreshed depth at every cell.

This is an appearance-change heuristic, not a guarantee that unchanged RGB implies unchanged geometry. Slowly accumulating changes, camera motion and pseudo-depth errors can affect the representation.

Equation 10
mt,i=𝟏[cos(xt,i,xt−1,i)<0.996],bt,i={dt,i,mt,i=1,bt−1,i,mt,i=0.m_{t,i}=\mathbf{1}\left[\cos(x_{t,i},x_{t-1,i})<0.996\right],\qquad b_{t,i}=\begin{cases}d_{t,i},&m_{t,i}=1,\\ b_{t-1,i},&m_{t,i}=0.\end{cases} (10)

5.2 Training

Post-training.

Depth post-training starts from the same discrete checkpoint and uniformly mixes action-only, depth-only and depth-plus-action examples. The joint style supplies depth tokens to the continuous expert while still masking target action tokens. It follows the standard post-training compute and optimization recipe with additional depth-token embeddings.

Fine-tuning.

LIBERO Think fine-tuning equally mixes action-only and depth-plus-action examples, corrupts 10% of teacher-forced depth inputs with random codes, and learns a per-layer gate on depth keys/values. The gate is derived from non-depth context and initialized with bias −4, initially limiting depth influence. The combined recipe addresses imperfect generated depth rather than assuming oracle depth at deployment.

Equations 11–12
cℓ=∑tAt​(1−Mt)​Vℓ,tvlm∑tAt​(1−Mt),gℓ=σ⁡(wℓ⊤​cℓ+bℓ),c_{\ell}=\frac{\sum_{t}A_{t}(1-M_{t})V^{\mathrm{vlm}}_{\ell,t}}{\sum_{t}A_{t}(1-M_{t})},\qquad g_{\ell}=\sigma(w_{\ell}^{\top}c_{\ell}+b_{\ell}), (11)
K¯ℓ,tvlm=(1−Mt+Mt​gℓ)​Kℓ,tvlm,V¯ℓ,tvlm=(1−Mt+Mt​gℓ)​Vℓ,tvlm.\bar{K}^{\mathrm{vlm}}_{\ell,t}=\left(1-M_{t}+M_{t}g_{\ell}\right)K^{\mathrm{vlm}}_{\ell,t},\qquad\bar{V}^{\mathrm{vlm}}_{\ell,t}=\left(1-M_{t}+M_{t}g_{\ell}\right)V^{\mathrm{vlm}}_{\ell,t}. (12)

5.3 Adaptive depth inference

Inference pipeline.

The first query generates all 100 codes; subsequent queries replay unchanged spans and greedily decode changed cells. Both replayed and new tokens enter the current VLM context before the action expert runs. Cached depth is therefore reused as token content, while the model still processes it to build the current context.

Inference optimization.

The adaptive scheduler remains eager because update patterns vary. A static cache and CUDA Graph stages accelerate fixed transformer work around attention; attention itself stays eager as its effective key/value length changes. This explains why graph replay benefits Think less than the regular action loop.

Complete original PDF table S5.T3
Table 3: Embodied reasoning results. We evaluate on benchmarks across spatial reasoning, pointing, and embodied QA tasks. The best-performing open-weight model on each benchmark is in bold, and the second best is underlined.
Original selectable table
Models/Benchmarks
Point-Bench (Cheng et al., 2025)
RefSpatial (Zhou et al., 2025)
RoboSpatial-Poi (Song et al., 2025)
Where2Place (Yuan et al., 2024)
EmbSpatial (Du et al., 2024)
MindCube (Luo et al., 2026)
RoboSpatial-VQ (Song et al., 2025)

Overall Avg.

API call only
GR-ER 1.5 Thinking (Team et al., 2025b) 71.6 48.5 31.1 59.0 57.8 84.3 54.8 78.4 54.7 79.3 76.7 55.0 45.8 61.3
GR-ER 1.5 (Team et al., 2025b) 73.3 41.8 25.3 48.0 65.2 83.6 47.0 73.4 47.7 57.0 62.0 50.5 39.9 55.0
Gemini 2.5 Pro (Comanici et al., 2025) 62.7 33.6 8.3 37.0 69.2 85.9 56.0 78.0 59.2 71.3 74.7 55.0 51.1 57.1
GPT-5 (Singh et al., 2025) 43.6 23.5 19.0 37.0 71.3 86.1 59.0 81.5 58.0 69.3 86.7 64.4 52.9 57.9
GPT-5-mini (Singh et al., 2025) 39.5 23.0 12.5 33.5 56.4 85.9 57.3 78.8 55.6 70.7 81.3 59.2 46.2 53.8
Open weights only
Qwen3-VL-4B (Yang et al., 2025a) 63.8 49.5 62.3 63.0 65.2 85.8 39.5 78.1 27.4 62.7 60.7 47.6 61.4 59.0
Qwen3-VL-8B (Yang et al., 2025a) 64.2 47.5 56.6 63.0 66.6 85.6 41.5 78.2 33.4 73.7 70.0 49.3 63.3 61.0
LLaVA-OV-7B (Li et al., 2024a) 9.4 3.0 0.0 19.0 44.2 70.3 39.8 69.7 45.6 68.9 58.0 36.2 31.2 38.1
InternVL3.5-4B (Wang et al., 2025a) 43.0 21.5 23.0 53.0 58.7 80.5 37.0 72.0 42.6 61.0 49.3 44.6 56.5 49.4
InternVL3.5-8B (Wang et al., 2025a) 45.5 35.0 27.0 57.0 59.1 80.9 41.0 74.7 39.7 62.7 58.7 43.5 56.4 52.4
Molmo family: Open weights, Open data, Open code
Molmo2 (Clark et al., 2026) 76.9 52.5 13.9 18.0 50.8 64.4 46.3 67.0 37.6 57.9 50.0 46.9 26.1 46.8
Molmo2-ER 77.3 52.5 32.0 54.0 72.5 87.8 46.8 78.8 57.0 73.4 78.0 44.7 74.5 63.8

6 Experiments

Evaluation covers embodied VLM reasoning, embodiment-specific zero-shot transfer, simulation and hardware fine-tuning, perturbation robustness, trajectory diagnostics and speed. The tables use different outcome definitions and uncertainty conventions, so their percentages should be interpreted within each protocol rather than combined into one reliability claim.

Complete original PDF table S6.T4
Table 4: Results across MolmoSpace tasks. We evaluate all models on four manipulation skill categories—Pick, Pick & Place, Open, and Close—and report mean success rate (%) with standard error across three independent evaluation runs. Bold denotes the best and underline the second-best per task. MolmoAct2-DROID achieves the highest overall average (37.7), outperforming the strongest prior baseline π0.5\pi_{0.5}-DROID (34.5) by +3.2 points, with particularly large gains on Pick (+7.3) and Pick & Place (+13.1), where contact-rich, multi-stage reasoning is required. On Close, MolmoAct2-DROID also sets a new best at 70.8, while on Open it trails the leading methods, suggesting that articulated-object interaction remains a direction for further improvement. Standard errors remain comparable across methods (≤3.2\leq 3.2), indicating that the observed gains are stable across runs rather than artifacts of evaluation variance.
Original selectable table
Models/Tasks Pick Pick & Place Open Close Average
StereoVLA (Deng et al., 2025) 7.0±\pm2.3 – – – 7.0
LAP-VLA (Zha et al., 2026) 24.9±\pm2.7 6.6±\pm1.6 11.4±\pm1.5 45.9±\pm2.9 22.2
π0\pi_{0}-DROID (Black et al., 2024) 16.2±\pm2.6 12.5±\pm2.0 11.0±\pm2.0 53.1±\pm3.2 23.2
π0.5\pi_{0.5}-DROID (Intelligence et al., 2025) 36.4±\pm2.9 13.6±\pm2.2 22.7±\pm2.6 65.1±\pm3.1 34.5
MolmoAct2-DROID 43.7±\pm3.1 26.7±\pm2.8 9.5±\pm2.0 70.8±\pm3.0 37.7
Complete original PDF table S6.T5
Table 5: Evaluation on simulation held-out environments. Simulation success rates are evaluated over 1000 episodes per task. For pick-and-place tasks, we report both oracle success (first number, which is the success conditions being fulfilled at any timestep) and success at end (second number, the success conditions being fulfilled at the final timestep). The delta between captures both unstable/unsuitable placement and the inability of policies to determine when a specified task is already completed, e.g. by repeatedly picking up an object which has already been placed correctly. We additionally report the half-width of the 95% confidence interval bounds for each result. Bold denotes the best and underline the second-best per task.
Original selectable table
Models/Tasks Pick MSProc Pick Classic Pick Pick Rand.-Cam. Pick&Place PnP Next-To PnP Color Avg.
StereoVLA (Deng et al., 2025) 6.6±\pm2.6 4.3±\pm1.5 1.1±\pm1.0 - - - - -
X-VLA (Zheng et al., 2025) 3.3±\pm1.0 0.5±\pm0.5 0.7±\pm0.5 0.8±\pm0.5 0.1±\pm0.2/0.1±\pm0.2 1.9±\pm0.9/1.0±\pm0.7 0.9±\pm0.6/0.5±\pm0.5 1.2
LAP-VLA (Zha et al., 2026) 19.4±\pm2.4 2.4±\pm1.0 3.1±\pm1.1 2.7±\pm1.0 3.81±\pm1.5/1.59±\pm1.0 6.48±\pm2.8/3.41±\pm2.1 3.1±\pm1.1/1.5±\pm0.8 4.8
π0.5\pi_{0.5}-DROID (Intelligence et al., 2025) 18.1±\pm2.4 6.4±\pm1.5 7.0±\pm1.6 8.0±\pm1.9 11.7±\pm2.1/7.6±\pm1.7 8.2±\pm2.2/6.2±\pm1.9 10.4±\pm1.9/6.7±\pm1.6 10.0
MolmoAct2-DROID 35.6\mathbf{35.6}±\pm3.0 18.9\mathbf{18.9}±\pm2.6 20.5\mathbf{20.5}±\pm2.6 15.4\mathbf{15.4}±\pm2.4 15.8\mathbf{15.8}±\pm2.4/7.8\mathbf{7.8}±\pm1.8 20.9\mathbf{20.9}±\pm2.6/14.4\mathbf{14.4}±\pm2.3 17.2\mathbf{17.2}±\pm2.5/8.8\mathbf{8.8}±\pm2.0 20.6\mathbf{20.6}
Complete original PDF table S6.T6
Table 6: Success rates (%) across manipulation tasks. Each cell reports the percentage of successful trajectories out of 15 trials. Bold denotes the best and underline the second-best per task.
Original selectable table
Models/Tasks Apple on plate Pipette in tray Red cube in tape roll Knife in box Objects in bowl Average
π0.5\pi_{0.5}-DROID (Intelligence et al., 2025) 66.7 33.3 53.3 26.7 46.2 45.2
MolmoBot (Deshpande et al., 2026) 86.7 53.3 33.3 40.0 28.6 48.4
MolmoAct2-DROID 100.0 86.7 93.3 93.3 62.0 87.1
Complete original PDF table S6.T7
Table 7: Success rates (%) across manipulation tasks on the SO-100 platform. Each cell reports the mean score over 15 trials, where partial credit is awarded for sub-task completion (0.25 for reaching, 0.5 for pickup, 1.0 for successful placement). Bold denotes the best and underline the second-best per task.
Original selectable table
Models/Tasks Fork on plate Stack blocks Tissues in basket Pen on notebook Block in box Average
SmolVLA (Shukor et al., 2025) 3.3 5.0 0.0 3.3 0.0 2.3
π0\pi_{0}-SO100/101 (Chen, 2025) 30.0 6.7 20.0 80.0 90.0 45.3
MolmoAct2-SO100/101 70.0 20.0 73.3 86.7 33.3 56.7
Complete original PDF figure S6.F6
Figure 6: Performance comparison on the RoboEval benchmark. (A) Task-wise success rates (%) across eight manipulation tasks for Diffusion Policy, GR00T N1.5, X-VLA, π0.5\pi_{0.5}-DROID, and MolmoAct. MolmoAct achieves the highest or near-highest success rates on most tasks, with particularly strong performance on long-horizon tasks such as Pack Box and Rotate Valve. π0.5\pi_{0.5}-DROID is the strongest baseline overall, while other methods exhibit lower and more variable performance across tasks. (B) Radar plot of normalized performance across behavioral and outcome metrics, averaged over all tasks (outer edge indicates best performance per metric). MolmoAct consistently dominates across nearly all metrics, indicating improvements not only in task success but also in efficiency, stability, and trajectory quality. Abbreviations: CT = completion time, TL = trajectory length, JPL = joint path length, CPL = Cartesian path length, CJ = Cartesian jerk, JJ = joint jerk, SC = self-collisions, SL = slip count.
Complete original PDF table S6.T8
Table 8: LIBERO benchmark success rates across four task categories (Spatial, Object, Goal, and Long-horizon) along with the average performance. MolmoAct2 achieves the highest overall average success rate of 97.2%, outperforming all strong baselines, with strong performance across all categories, particularly scoring 100% on the LIBERO-Object tasks. Furthermore improvement could be seem going from MolmoAct2 to MolmoAct2-Think. Bold denotes the best and underline the second-best per task.
Original selectable table
Baseline Spatial Object Goal Long Average
TraceVLA (Zheng et al., 2024) 84.6% 85.2% 75.1% 54.1% 74.8%
OpenVLA (Kim et al., 2024) 84.7% 88.4% 79.2% 53.7% 76.5%
SpatialVLA (Qu et al., 2025) 88.2% 89.9% 78.6% 55.5% 78.1%
CoT-VLA (Zhao et al., 2025) 87.5% 91.6% 87.6% 69.0% 83.9%
π0\pi_{0} (Black et al., 2024) 96.8% 98.8% 95.8% 85.2% 94.2%
ThinkAct (Huang et al., 2025) 88.3% 91.4% 87.1% 70.9% 84.4%
MolmoAct-7B-D (Lee et al., 2025) 87.0% 95.4% 87.6% 77.2% 86.6%
GR00T N1.7 (NVIDIA et al., 2025) 97.7% 97.5% 98.5% 94.4% 97.0%
π0.5\pi_{0.5} (Intelligence et al., 2025) 98.8% 98.2% 98.0% 92.4% 96.9%
NORA-1.5 (Hung et al., 2025) 97.3% 96.4% 94.5% 89.6% 94.5%
MolmoAct2 97.8% 100.0% 97.8% 93.2% 97.2%
MolmoAct2-Think 98.8% 99.8% 98.5% 95.4% 98.1%
Complete original PDF figure S6.F7
Figure 7: Overview of efficient fine-tuning of MolmoAct2. We conduct a comprehensive evaluation across 8 real-world tasks outside of a laboratory setting, ranging from preparing a pipette for a chemist to placing toys back onto a shelf. MolmoAct2, fine-tuned on the collected evaluation data, outperforms 4 strong baselines by a large margin of 15% over the runner-up model.

6.1 Molmo2-ER evaluation

Evaluation setup.

Thirteen columns cover pointing, spatial VQA, cross-view reasoning and video understanding. RoboSpatial contributes separate pointing and VQA columns, explaining the difference between the number of named benchmark families and the table’s column count.

Baselines.

Comparisons include API-only Gemini/GPT models, open-weight Qwen/LLaVA/InternVL models and Molmo2. The table’s boldface marks open-weight leadership, not necessarily the best score across API and open models.

Results.

Molmo2-ER has the best reported mean, 63.8 versus 61.3 for Gemini-ER 1.5 Thinking and 46.8 for Molmo2: a 17-point improvement over its initialization. Table 3 gives four outright all-model wins and one RefSpatial tie. Nine leading results including that tie apply to the open-weight comparison, not nine outright wins over all models. It trails closed models on several embodied QA columns and even its initialization on OpenEQA.

6.2 Out-of-the-box deployment

The DROID checkpoint transfers without benchmark-specific retraining to DROID-like simulation and hardware. MolmoSpaces averages 37.7 versus 34.5 for π0.5-DROID, but Open is worse at 9.5 versus 22.7. MolmoBot averages 20.6 versus 10.0 using the table’s oracle pick-and-place successes; end-of-rollout successes are substantially lower, such as 7.8 versus 15.8 oracle on ordinary pick-and-place.

Hardware uses five tasks and fifteen trials each. DROID’s reported mean is 87.1 versus 48.4, but its objects-in-bowl task uses partial credit. SO’s 56.7 versus 45.3 is entirely a milestone-weighted mean, and block-in-box drops to 33.3 versus the baseline’s 90.0. Cameras are initially placed outside training layouts and then fixed across compared policies, not necessarily moved randomly every trial. These are gains on known embodiments, not universal zero-shot control. The source’s broad statistical-significance claim is not established for every comparison by trial counts alone.

6.3 Effective fine-tuning

Standard MolmoAct2 reaches 97.2 on LIBERO versus 97.0 for the strongest listed external baseline, and 44.3 on RoboEval versus about 40.5. Hardware adaptation trains separate single-task policies and evaluates eight YAM tasks with fifty episodes each; seven task rows favor MolmoAct2, while cup stacking favors another model.

The main text reports a 50.1 hardware mean, whereas Appendix Table 19 reports 50.6 and its rows average about 50.56. The task naming also differs: the appendix calls the 64-point row Linearbot while the main description names make_popcorn. Both original presentations are preserved rather than silently reconciled. These results demonstrate useful adaptation, but roughly half the evaluated hardware tasks still fail on average.

6.4 Performance of MolmoAct2-Think

Think improves LIBERO’s mean from 97.2 to 98.1, with Long rising from 93.2 to 95.4. Table 8 shows Object slipping from 100.0 to 99.8, despite the prose saying it matches; Goal is 98.5 there but 98.4 in Table 14. The gains are small and use best selected checkpoints. Two thousand evaluation rollouts without repeated training seeds or a reported paired significance analysis do not by themselves prove the improvement is non-incidental.

Complete original PDF table S6.T9
Table 9: Robustness of MolmoAct2 under environmental perturbations. We evaluate MolmoAct2 against four baselines on out-of-distribution scenarios spanning spatial variation, lighting, language rephrasing, and visual distractors. MolmoAct2 is the most robust, achieving an average success rate of 50.69% across all perturbation types—a 10.80% absolute improvement over the next-best model. Values report average success rates (%); bold = best, underline = second-best.
Original selectable table
Model Spatial Var. Lighting Language Distractor Overall
π0.5\pi_{0.5} (Intelligence et al., 2025) 15.00 33.70 26.15 33.20 27.01
Cosmos Policy (Kim et al., 2026a) 8.75 17.50 5.00 13.75 11.25
X-VLA (Wen et al., 2025) 3.75 8.30 9.95 3.75 6.44
OpenVLA-OFT (Kim et al., 2025) 13.75 46.25 51.25 48.30 39.89
MolmoAct2-Think 26.25 62.05 60.35 54.10 50.69

6.5 Robustness of MolmoAct2 to out-of-distribution shifts

Four hardware tasks are tested under spatial, lighting, instruction and distractor changes. The reported overall score is 50.69 versus 39.89 for OpenVLA-OFT, with the weakest category being spatial variation at 26.25. Table 9 labels the proposed row MolmoAct2-Think while the surrounding prose names MolmoAct2, leaving the exact variant ambiguous.

The stated schedule is twenty trials per task, five per perturbation; several fractional scores are not simple binary increments under that count, and the aggregation/partial-credit details are not fully specified. The result supports the reported comparison, not a precise uncertainty claim or general robustness guarantee.

6.6 Quality of MolmoAct2 trajectories for real-world deployment

RoboEval supplements success with path length, jerk, collisions, slipping and bimanual coordination. The paper highlights shorter completion times, including 4.70 seconds on Stack Two Blocks and 8.51 on Rotate Valve. The full combined-variation table reveals task-specific tradeoffs, so a favorable normalized radar plot should not be read as dominance on every raw metric or as a hardware safety certificate.

6.7 Systematic analysis

Ablations change the backbone, conditioning interface, number of flow samples, fine-tuning updates and depth regularization. Some start directly from the discrete checkpoint and therefore are not interchangeable with the final model’s full pretraining/post-training/fine-tuning pipeline.

Embodied-reasoning backbone.

With discrete actions only and 60,000 training steps on LIBERO Long, replacing Molmo2 by Molmo2-ER raises success from 77.6 to 83.6. This supports transfer of embodied specialization into action-token prediction under the matched setup.

Complete original PDF table S6.T10
Table 10: Backbone ablation on LIBERO Long. Both variants are trained directly from the VLM checkpoint with only discrete action prediction using MolmoAct2-FAST Tokenizer. The final row is marked in pink, which is what we initialize with for MolmoAct2-Pretrain.
Original selectable table
Backbone Action objective Long
Molmo2 (Clark et al., 2026) Discrete 77.6%
Molmo2-ER Discrete 83.6%

VLM-to-expert conditioning.

Conditioning on flattened per-layer keys/values averages 95.9, versus 94.8 for a head-preserving variant and 94.0 for hidden-state conditioning. The standard design is strongest on average, while hidden states remain best on Spatial. The result supports this tested interface rather than a universal claim about every VLM/expert architecture.

Complete original PDF table S6.T11
Table 11: Ablation of the VLM-to-action-expert conditioning source on LIBERO. All variants are trained from MolmoAct2-Pretrain with the same configuration. Bold denotes the best result per column. The final row is marked in pink, which is what we use in all MolmoAct2 and MolmoAct2-Think post-training and fine-tuning runs.
Original selectable table
Conditioning source Spatial Object Goal Long Average
Hidden-state conditioning 97.0% 95.4% 96.6% 87.0% 94.0%
Per-head per-layer KV conditioning 95.8% 97.2% 96.6% 89.6% 94.8%
Per-layer KV conditioning 96.2% 99.0% 98.6% 89.8% 95.9%

Multiple flow samples.

Increasing flow samples from one to eight improves the four-suite mean from 94.15 to 95.90. Per-suite gains are not monotonic: two samples score 92.2 on Long versus 89.8 with eight. Additional samples also add expert computation.

Complete original PDF table S6.T12
Table 12: Ablation of the number of flow samples on LIBERO. All rows use per-layer KV conditioning and are trained from MolmoAct2-Pretrain with the same configuration. Bold denotes the best result per column. The final row is marked in pink, which is what we stick to in all of MolmoAct2 and MolmoAct2-Think fine-tuning runs.
Original selectable table
KK Spatial Object Goal Long Average
1 95.0% 95.8% 98.2% 87.6% 94.15%
2 96.6% 96.0% 95.4% 92.2% 95.05%
4 96.2% 98.4% 97.0% 89.0% 95.15%
8 96.2% 99.0% 98.6% 89.8% 95.90%

Fine-tuning design.

The final full fine-tune averages 97.20, versus 96.95 without discrete co-training, 97.05 with knowledge insulation, 96.25 with LoRA and 93.05 when only the expert learns. The small differences among full-model variants lack repeated-seed uncertainty, while the expert-only gap is larger. Removing co-training improves Long despite lowering the overall mean.

Complete original PDF table S6.T13
Table 13: Ablation of MolmoAct2 fine-tuning design choices on LIBERO. Component columns indicate whether each design choice is enabled: ✓ denotes enabled and ✗ denotes disabled. Bold denotes the best result per column. The final row is the MolmoAct2-LIBERO recipe, where its training design is marked in pink, which is what we stick to in all of MolmoAct2 fine-tuning runs.
Original selectable table
Discrete co-training Knowledge insulation Training type Spatial Object Goal Long Average
✗ ✗ Full fine-tuning 95.8% 98.6% 98.4% 95.0% 96.95%
✓ ✓ Full fine-tuning 96.8% 99.4% 97.2% 94.8% 97.05%
✓ ✗ LoRA 98.0% 99.4% 97.2% 90.4% 96.25%
✗ ✗ Action expert only 91.6% 97.6% 93.8% 89.2% 93.05%
✓ ✗ Full fine-tuning 97.8% 100.0% 97.8% 93.2% 97.20%

Depth-aware fine-tuning.

The combined depth-noise/gate/mixed-training recipe gives 98.10, versus 97.65 without noise and gating and 97.50 without all three. Noise and gating are removed together, so this table cannot isolate their individual contributions. Long is actually highest in the unregularized depth-only row.

Complete original PDF table S6.T14
Table 14: Ablation of MolmoAct2-Think depth fine-tuning choices on LIBERO. Mixed training denotes uniform sampling of action-only and depth-and-action examples. ✓ denotes enabled and ✗ denotes disabled. Bold denotes the best result per column. The final all-enabled row is the MolmoAct2-Think-LIBERO recipe, where its training design is marked in pink, which is what we stick to in all of MolmoAct2-Think fine-tuning runs.
Original selectable table
Mixed training Noise injection Depth gate Spatial Object Goal Long Average
✗ ✗ ✗ 97.6% 99.6% 96.6% 96.2% 97.50%
✓ ✗ ✗ 98.8% 99.8% 96.6% 95.4% 97.65%
✓ ✓ ✓ 98.8% 99.8% 98.4% 95.4% 98.10%

6.8 Inference speed

On one H100 with horizon ten, caching plus CUDA Graphs raises action-only throughput from 23.02 to 55.79 actions/s and Think from 8.04 to 12.71. These correspond to roughly 179 ms and 787 ms per generated chunk, not observation-feedback frequencies of 55.79 and 12.71 Hz. The optimized discrete paths are slower at 14.17 and 6.82 actions/s.

Appendix E states that full one-second chunks execute open-loop before re-querying. No within-chunk correction or boundary-continuity objective is provided, so high generation throughput should not be confused with rapid closed-loop reaction.

Complete original PDF figure S6.F8
Figure 8: Inference control rate after caching and CUDA Graph optimizations. Control rate is action horizon divided by end-to-end action-generation latency; measurements use horizon 10 on LIBERO with a single H100.

7 Related work

Generalist robot manipulation policies.

Generalist policies transfer visual/language representations into robot imitation learning across datasets and embodiments. This work emphasizes releasing training ingredients as well as weights, while its demonstrated deployment coverage remains concentrated on the trained platforms.

Embodied reasoning for robotic manipulation.

Embodied reasoning may use points, boxes, future images, trajectories or latent states. Adaptive depth extends MolmoAct by reusing stable scene regions. The representation is interpretable as predicted geometry, but no causal faithfulness test establishes that each displayed depth token explains the resulting action.

Bridging VLM and action expert.

The architecture joins discrete autoregressive training with a continuous expert. Its distinguishing interface exposes corresponding-layer keys/values instead of only final hidden states. Depth reasoning changes the expert’s context, while flow matching produces the entire continuous chunk.

8 Conclusion

The contribution is a reproducible family of robot models and data with strong reported transfer and fine-tuning results. Practical limits remain substantial: embodiment-specific data dependence, low absolute success on some tasks, partial-credit metrics, expensive training and open-loop chunk execution. The complete source artifacts make those strengths and limits inspectable.

Author contributions

Haoquan Fang and Jiafei Duan share first authorship. The contribution statement credits specialist leads for tokenizer, data curation, backbone, depth reasoning and inference, along with hardware, evaluation, infrastructure and advisory teams. Ranjay Krishna is identified as project PI.

Acknowledgment

The authors acknowledge Ai2 colleagues and cluster support, Cirrascale infrastructure and Jiafei Duan’s A*STAR National Science Fellowship.

Appendix

The appendices supply architecture, training implementation, evaluation protocols, data statistics and explicit deployment limitations. All implementation listings, prompts and supplementary visual results are retained.

Appendix A Model Details

The model first learns a discrete robot interface, then gains a continuous expert. Think adds a depth prefix without replacing the backbone/expert structure. The following details specify the released architecture rather than a separate experimental system.

A.1 MolmoAct2 Backbone

Backbone initialization and visual inputs.

Robot observations use a single resized crop per camera to control sequence length. Video input is capped at eight frames and 2 FPS; camera serialization is randomized during pretraining. This trades fine visual detail for manageable multi-view context.

Vision encoder and connector.

SigLIP2 features from the third-to-last and ninth-to-last layers enter attention pooling: 2×2 windows for images and 3×3 for videos. A shared MLP maps pooled features into the language-model embedding space.

Language model input.

Image-index text and frame markers identify visual inputs. Visual tokens can attend across views and frames before action generation. The robot pipeline uses single crops rather than the base model’s tiled-image column machinery.

Added tokens.

Setup/control/state delimiters and output triggers distinguish robot semantics from task language. The vocabulary adds 2,048 action tokens and 256 state tokens; Think adds 128 depth codes plus its trigger/delimiters. The appendix’s illustrative DROID delta-end-effector prompt should not override the dedicated DROID checkpoint’s stated absolute-joint control.

Table 15 lists a 380M vision encoder, 57M connector, 4B language model and 621M action expert. The expert width is 768 with 36 layers and eight attention heads; architecture size is larger than the language-model-only 4B label.

Complete original PDF table A1.T15
Table 15: MolmoAct2 architecture hyperparameters. The image encoder, connector, and LLM columns follow the 4B Molmo2-ER backbone used by MolmoAct2. The action-expert column describes the MolmoAct2 continuous flow-matching module used for continuous action prediction.
Original selectable table
Hyperparameter Image Encoder V/L Connector LLM Action Expert
Params 380M 57M 4.0B 621M
Dim 1152 1152 2560 768
MLP Dim 4304 9728 9728 3072
Act. GELU SwiGLU SwiGLU SwiGLU
Heads 16 16 32 8
KV Heads 16 – 8 –
Layers 27 – 36 36
Dropout 0.0 0.0 0.1 0.0
Image Size 384×384384{\times}384 – – –
Patch Size 14 – – –
Image Pool Size – 2×22{\times}2 – –
Video Pool Size – 3×33{\times}3 – –
Pool Dim – 1152 – –
Pool Heads – 16 – –
Embed – – 151936 –
Theta – – 1M –
Max Horizon – – – 30
Max Action Dim – – – 32
Time Embed Dim – – – 256
Conditioning – – – Per-layer KV

A.2 Continuous Action Expert

Input and output.

The expert is a noncausal transformer over the noisy horizon-by-32 action tensor. It uses horizon thirty in post-training and ten for LIBERO, with padding masks removing unused coordinates and time steps.

Flow-matching objective.

The exact flow equations retain noise-to-data interpolation and masked squared-error training. Released inference uses ten Euler steps, then slices the output to the embodiment’s action width and reverses its normalization. Numerical integration and imitation learning provide no explicit action-safety constraint.

Equations 13–15
xt=(1−t)​ϵ+t​a,u⋆=a−ϵ.x_{t}=(1-t)\epsilon+ta,\qquad u^{\star}=a-\epsilon. (13)
ℒflow=𝔼a,ϵ,t​[‖m⊙(fθ​(xt,t,c)−u⋆)‖22],\mathcal{L}_{\mathrm{flow}}=\mathbb{E}_{a,\epsilon,t}\left[\left\|m\odot\left(f_{\theta}(x_{t},t,c)-u^{\star}\right)\right\|_{2}^{2}\right], (14)
zi+1=zi+Δ​t​fθ​(zi,ti,c),ti=iN,Δ​t=1N.z_{i+1}=z_{i}+\Delta t\,f_{\theta}(z_{i},t_{i},c),\qquad t_{i}=\frac{i}{N},\qquad\Delta t=\frac{1}{N}. (15)

Expert block.

Each of 36 blocks combines bidirectional action attention, visual-language cross-attention and an MLP. Time embeddings modulate RMS normalization and residual gates, query/key normalization stabilizes attention, and rotary embeddings order the action sequence. The output projection begins near zero velocity.

Equations 16–18
hℓ′\displaystyle h^{\prime}_{\ell} =hℓ+gℓsa​SA⁡(AdaRMSℓsa⁡(hℓ,t)),\displaystyle=h_{\ell}+g^{\mathrm{sa}}_{\ell}\operatorname{SA}\!\left(\operatorname{AdaRMS}^{\mathrm{sa}}_{\ell}(h_{\ell},t)\right), (16)
h¯ℓ\displaystyle\bar{h}_{\ell} =hℓ′+gℓca​CA⁡(AdaRMSℓca⁡(hℓ′,t),K~ℓ,V~ℓ),\displaystyle=h^{\prime}_{\ell}+g^{\mathrm{ca}}_{\ell}\operatorname{CA}\!\left(\operatorname{AdaRMS}^{\mathrm{ca}}_{\ell}(h^{\prime}_{\ell},t),\tilde{K}_{\ell},\tilde{V}_{\ell}\right), (17)
hℓ+1\displaystyle h_{\ell+1} =h¯ℓ+gℓff​MLP⁡(AdaRMSℓff⁡(h¯ℓ,t)).\displaystyle=\bar{h}_{\ell}+g^{\mathrm{ff}}_{\ell}\operatorname{MLP}\!\left(\operatorname{AdaRMS}^{\mathrm{ff}}_{\ell}(\bar{h}_{\ell},t)\right). (18)

Per-layer KV conditioning.

The VLM supplies eight 128-wide key/value heads, flattened to 1,024 dimensions per token. Learned adapters map them to the expert’s 768 dimensions, arranged as eight 96-wide heads. Detachment during post-training applies to this continuous-loss path, not to all backbone learning.

Equations 19–20
K~ℓ=reshape⁡(PK​Kℓvlm),V~ℓ=reshape⁡(PV​Vℓvlm).\tilde{K}_{\ell}=\operatorname{reshape}\!\left(P_{K}K^{\mathrm{vlm}}_{\ell}\right),\qquad\tilde{V}_{\ell}=\operatorname{reshape}\!\left(P_{V}V^{\mathrm{vlm}}_{\ell}\right). (19)
CA⁡(Qℓ,K~ℓ,V~ℓ)=softmax⁡(Qℓ​K~ℓ⊤dh)​V~ℓ.\operatorname{CA}(Q_{\ell},\tilde{K}_{\ell},\tilde{V}_{\ell})=\operatorname{softmax}\!\left(\frac{Q_{\ell}\tilde{K}_{\ell}^{\top}}{\sqrt{d_{h}}}\right)\tilde{V}_{\ell}. (20)

A.3 Adaptive-Depth Extension

Depth tokens form a 100-position prefix generated before actions. A learned sigmoid gate, computed from non-depth context, scales depth keys and values separately at each layer. Bias −4 initially suppresses their influence, and fine-tuning learns how much depth context to use.

Equations 21–22
cℓ=∑tAt​(1−Mt)​Vℓ,tvlm∑tAt​(1−Mt),gℓ=σ⁡(wℓ⊤​cℓ+bℓ),c_{\ell}=\frac{\sum_{t}A_{t}(1-M_{t})V^{\mathrm{vlm}}_{\ell,t}}{\sum_{t}A_{t}(1-M_{t})},\qquad g_{\ell}=\sigma(w_{\ell}^{\top}c_{\ell}+b_{\ell}), (21)
K¯ℓ,tvlm=(1−Mt+Mt​gℓ)​Kℓ,tvlm,V¯ℓ,tvlm=(1−Mt+Mt​gℓ)​Vℓ,tvlm.\bar{K}^{\mathrm{vlm}}_{\ell,t}=\left(1-M_{t}+M_{t}g_{\ell}\right)K^{\mathrm{vlm}}_{\ell,t},\qquad\bar{V}^{\mathrm{vlm}}_{\ell,t}=\left(1-M_{t}+M_{t}g_{\ell}\right)V^{\mathrm{vlm}}_{\ell,t}. (22)

Appendix B Training Details

B.1 Implementation

Training infrastructure.

Training uses PyTorch FSDP2, compiled static shapes and scaled-dot-product attention with custom masks. Most operations use bfloat16; normalization and rotary-position calculations retain full precision. Token losses normalize using the mean target-token count across devices, avoiding disproportionate weighting of short targets. The appendix says fewer than 0.1% of overlong examples are truncated, whereas the main recipe describes skipping them.

Packing.

Each loader worker pools 48 examples and solves a token/image packing problem with image weight thirty, rounding token counts to multiples of 32. Masks preserve independence among packed examples. Generic Molmo2 sequence limits listed here should not be confused with MolmoAct2’s stage-specific 4,200/2,100-token budgets.

Equation 23
T+wi​Isubject toT≤Tmax,I≤Imax,T+w_{i}I\quad\text{subject to}\quad T\leq T_{\max},\ I\leq I_{\max}, (23)

Image augmentation.

The exact augmentation listing applies a 95% crop, resize, ±5-degree rotation, color jitter and Gaussian blur with probability 0.2. It applies to training images/videos and is disabled at evaluation. The source code is preserved alongside its native PDF rendering.

Complete original PDF listing augmentation-code
Original image-augmentation code
Original selectable listing
import torchvision.transforms as T
height, width = image.height, image.width
transform = T.Compose([
T.RandomCrop(size=(int(height * 0.95), int(width * 0.95))),
T.Resize((height, width)),
T.RandomRotation(degrees=5),
T.ColorJitter(
brightness=0.2,
contrast=(0.8, 1.2),
saturation=(0.8, 1.2),
hue=0.05,
),
T.RandomApply(
[T.GaussianBlur(kernel_size=5, sigma=(0.1, 1.0))],
p=0.2,
),
])
image = transform(image)

Input prompts.

The three exact templates select action, depth or depth-then-action output using assistant-side triggers. Task text is normalized, while setup, control mode and optional state tokens occupy distinct fields. The same chat prefix is used at training and inference. All delimiters and example token sequences are retained.

Complete original PDF listing robot-prompts
Original robot prompt templates
Original selectable listing
Action:
<|im_start|>user
The task is to put the red mug on the tray. The setup is <setup_start>bimanual yam robotic arms in molmoact2<setup_end>. The current state of the robot is <state_start><state_14><state_87>...<state_203><state_end>. The expected control mode is <control_start>absolute joint pose<control_end>. Given these, what action should the robot take to complete the task?
<|im_end|>
<|im_start|>assistant
<action_output>
Depth:
<|im_start|>user
The task is to open the middle drawer. The setup is <setup_start>single franka robotic arm in libero<setup_end>. The expected control mode is <control_start>delta end-effector pose<control_end>. Given these, what is the depth map of the main image?
<|im_end|>
<|im_start|>assistant
<depth_output>
Depth then action:
<|im_start|>user
The task is to place the bowl next to the plate. The setup is <setup_start>single franka robotic arm in libero<setup_end>. The current state of the robot is <state_start><state_31><state_52>...<state_118><state_end>. The expected control mode is <control_start>delta end-effector pose<control_end>. Given these, first predict the depth map of the main image and then predict the action the robot should take to complete the task?
<|im_end|>
<|im_start|>assistant
<depth_output><action_output>

Training hyperparameters.

Table 16 records stage hyperparameters, including 200 warm-up steps, optimizer ε=10⁻⁶ and β=(0.9,0.95), learning rates and compute. Its pretraining action-expert entries are template values: the method explicitly introduces that expert only at post-training. Fine-tuning figures describe major embodiment runs; LIBERO and smaller evaluations have separate schedules.

Complete original PDF table A2.T16
Table 16: MolmoAct2 training hyperparameters. We summarize the stage-level hyperparameters used for pre-training, post-training, and embodiment-specific fine-tuning. We only show the fine-tuning hyperparameters for the training on major datasets.
Original selectable table
Hyperparameter Pre-train Post-train Fine-tune
Warm-up ViT 200 200 200
Warm-up Conn. 200 200 200
Warm-up LLM 200 200 200
Warm-up AE 200 200 200
LR ViT 5×10−65{\times}10^{-6} 5×10−65{\times}10^{-6} 5×10−65{\times}10^{-6}
LR Conn. 5×10−65{\times}10^{-6} 5×10−65{\times}10^{-6} 5×10−65{\times}10^{-6}
LR LLM 1×10−51{\times}10^{-5} 1×10−51{\times}10^{-5} 1×10−51{\times}10^{-5}
LR AE 5×10−55{\times}10^{-5} 5×10−55{\times}10^{-5} 5×10−55{\times}10^{-5}
Cosine Decay 10% 10% 10%
Eps. 10−610^{-6} 10−610^{-6} 10−610^{-6}
Betas 0.9/0.95 0.9/0.95 0.9/0.95
Steps 200k 100k 100k
Global Batch Size 128 128 64 or 128 (BimanualYAM)
Sequence Length 4200 2100 (Robot) and 4200 (Multimodal) 2100
GPUs (H100s) 64 64 32 or 64 (BimanualYAM)
Time (Hours) 90 36 36
GPU Hours 5760 2304 1152 or 2304 (BimanualYAM)

B.2 MolmoAct2-FAST Tokenizer

The tokenizer keeps FAST’s DCT/BPE compression logic and exposes its training mixture. Its openness makes the action vocabulary reproducible; it does not create a coordinate-invariant controller by itself.

Data Format

Single-arm vectors place arm coordinates and a gripper value before zero padding. Bimanual vectors concatenate left arm/gripper then right arm/gripper. The shared maximum is 32 dimensions.

Multimodal Representation

The tokenizer supports multiple control conventions rather than transforming all actions into one coordinate frame. Setup/control descriptors and visual context let the backbone distinguish absolute joint targets from end-effector commands.

B.3 GPU Cluster

Runs use Ai2’s Jupiter cluster, scheduled through Beaker and operated by Cirrascale. The cluster description gives infrastructure capacity, not the number of GPUs used by every experiment.

Compute

Jupiter has 1,024 H100 80GB GPUs across 128 servers, with dual Xeon CPUs, 2TB RAM and 18TB local NVMe per server. Individual training runs use the smaller GPU counts stated in their recipes.

Storage

Shared WEKA storage provides roughly 1PB NVMe plus 5PB HDD. The reported 761Gbps aggregate benchmark uses 64 clients; it is a cluster storage measurement rather than model training throughput.

Interconnect

A balanced rail-optimized InfiniBand network supports cross-node communication with eight 400Gbps adapters per server. This provides the infrastructure for multi-node training.

Cooling

Water-cooled closed-loop cabinets are reported to achieve datacenter PUE 1.2, with typical H100 temperatures of 60–65°C. These operational figures are not a measured energy or carbon accounting for this training project.

Appendix C Evaluation Details

C.1 RoboEval

RoboEval fine-tunes all eight tasks jointly for 10,000 steps on eight GPUs with batch 64. Evaluation varies position, orientation and task-specific conditions, and logs success, progress, path/jerk, contact and coordination metrics. Table 17 includes the full combined-variation results and human statistics from successful demonstrations only; that human row is not an identically sampled policy evaluation.

Complete original PDF table A3.T17
Table 17: Summary of performance metrics across all methods on the combined variation. Bold values indicate the best result for each metric within the task. Values are reported as mean ±\pm 95% confidence interval. The final row reports human demonstration statistics (computed from successful teleoperated demos only), which provide a reference band for efficient execution.
Original selectable table

Method Success ↑\uparrow TP ↑\uparrow BAVD ↓\downarrow BGVD ↓\downarrow CPL ↓\downarrow CT ↓\downarrow ECC ↓\downarrow JPL ↓\downarrow MCJ ↓\downarrow MJJ ↓\downarrow OPL ↓\downarrow SCC ↓\downarrow SC ↓\downarrow TL ↓\downarrow
Cube Handover
Diffusion 0.06± 0.100.06\,\pm\,0.10 0.27± 0.070.27\,\pm\,0.07 0.32± 0.030.32\,\pm\,0.03 0.13± 0.010.13\,\pm\,0.01 5.07± 0.355.07\,\pm\,0.35 4.80± 0.284.80\,\pm\,0.28 2.94± 0.492.94\,\pm\,0.49 26.14± 2.0226.14\,\pm\,2.02 24.73± 0.9024.73\,\pm\,0.90 133.00± 4.88133.00\,\pm\,4.88 19.61± 1.5919.61\,\pm\,1.59 2.46± 0.572.46\,\pm\,0.57 0.22± 0.140.22\,\pm\,0.14 939.76± 52.04939.76\,\pm\,52.04
GR00T 0.00± 0.070.00\,\pm\,0.07 0.01± 0.010.01\,\pm\,0.01 0.50± 0.010.50\,\pm\,0.01 0.10± 0.010.10\,\pm\,0.01 4.69± 0.264.69\,\pm\,0.26 9.52± 0.269.52\,\pm\,0.26 2.90± 0.492.90\,\pm\,0.49 33.15± 1.1133.15\,\pm\,1.11 42.62± 1.4942.62\,\pm\,1.49 235.44± 4.45235.44\,\pm\,4.45 23.34± 0.9723.34\,\pm\,0.97 25.88± 1.9125.88\,\pm\,1.91 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 976.10± 23.48976.10\,\pm\,23.48
xVLA 0.00± 0.070.00\,\pm\,0.07 0.04± 0.030.04\,\pm\,0.03 0.56± 0.040.56\,\pm\,0.04 0.15± 0.020.15\,\pm\,0.02 5.51± 0.685.51\,\pm\,0.68 8.50± 3.268.50\,\pm\,3.26 3.40± 0.593.40\,\pm\,0.59 31.99± 3.0131.99\,\pm\,3.01 31.00± 3.2031.00\,\pm\,3.20 146.84± 13.51146.84\,\pm\,13.51 23.25± 2.4723.25\,\pm\,2.47 13.68± 2.5313.68\,\pm\,2.53 0.04± 0.090.04\,\pm\,0.09 962.24± 42.87962.24\,\pm\,42.87
Pi_0.5 0.40± 0.14\mathbf{0.40}\,\pm\,0.14 0.65± 0.10\mathbf{0.65}\,\pm\,0.10 0.34± 0.030.34\,\pm\,0.03 0.05± 0.010.05\,\pm\,0.01 2.22± 0.442.22\,\pm\,0.44 3.66± 0.68\mathbf{3.66}\,\pm\,0.68 1.32± 0.47\mathbf{1.32}\,\pm\,0.47 11.22± 2.3111.22\,\pm\,2.31 18.14± 1.7818.14\,\pm\,1.78 88.53± 7.9888.53\,\pm\,7.98 8.00± 1.688.00\,\pm\,1.68 2.30± 0.992.30\,\pm\,0.99 0.26± 0.130.26\,\pm\,0.13 620.58± 120.85\mathbf{620.58}\,\pm\,120.85
MolmoAct 0.36± 0.140.36\,\pm\,0.14 0.58± 0.100.58\,\pm\,0.10 0.29± 0.030.29\,\pm\,0.03 0.04± 0.00\mathbf{0.04}\,\pm\,0.00 2.04± 0.312.04\,\pm\,0.31 4.21± 0.664.21\,\pm\,0.66 1.40± 0.441.40\,\pm\,0.44 10.78± 1.7310.78\,\pm\,1.73 18.43± 1.9118.43\,\pm\,1.91 90.54± 7.7390.54\,\pm\,7.73 7.99± 1.377.99\,\pm\,1.37 1.10± 0.43\mathbf{1.10}\,\pm\,0.43 0.26± 0.170.26\,\pm\,0.17 695.86± 111.27695.86\,\pm\,111.27
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.33± 0.030.33\,\pm\,0.03 0.03± 0.000.03\,\pm\,0.00 0.53± 0.030.53\,\pm\,0.03 0.51± 0.070.51\,\pm\,0.07 0.10± 0.160.10\,\pm\,0.16 2.26± 0.182.26\,\pm\,0.18 28.94± 3.0728.94\,\pm\,3.07 120.52± 11.54120.52\,\pm\,11.54 1.38± 0.181.38\,\pm\,0.18 0.17± 0.170.17\,\pm\,0.17 0.03± 0.130.03\,\pm\,0.13 103.53± 14.28103.53\,\pm\,14.28
Lift Pot
Diffusion 0.02± 0.080.02\,\pm\,0.08 0.07± 0.030.07\,\pm\,0.03 0.36± 0.050.36\,\pm\,0.05 0.18± 0.030.18\,\pm\,0.03 4.58± 0.564.58\,\pm\,0.56 8.87± 0.408.87\,\pm\,0.40 0.20± 0.180.20\,\pm\,0.18 24.46± 2.6324.46\,\pm\,2.63 27.14± 3.7727.14\,\pm\,3.77 142.55± 18.69142.55\,\pm\,18.69 16.55± 1.8816.55\,\pm\,1.88 10.64± 2.0210.64\,\pm\,2.02 0.12± 0.12\mathbf{0.12}\,\pm\,0.12 978.06± 32.64978.06\,\pm\,32.64
GR00T 0.06± 0.100.06\,\pm\,0.10 0.30± 0.050.30\,\pm\,0.05 0.51± 0.020.51\,\pm\,0.02 0.09± 0.020.09\,\pm\,0.02 2.13± 0.302.13\,\pm\,0.30 7.43± 1.227.43\,\pm\,1.22 4.54± 0.764.54\,\pm\,0.76 21.44± 2.6621.44\,\pm\,2.66 28.17± 1.9128.17\,\pm\,1.91 231.37± 6.96231.37\,\pm\,6.96 13.14± 1.6413.14\,\pm\,1.64 5.88± 2.265.88\,\pm\,2.26 1.34± 0.531.34\,\pm\,0.53 628.14± 92.83628.14\,\pm\,92.83
xVLA 0.00± 0.070.00\,\pm\,0.07 0.21± 0.050.21\,\pm\,0.05 0.67± 0.060.67\,\pm\,0.06 0.22± 0.040.22\,\pm\,0.04 5.31± 0.845.31\,\pm\,0.84 8.29± 3.488.29\,\pm\,3.48 2.26± 0.692.26\,\pm\,0.69 30.12± 3.7430.12\,\pm\,3.74 36.35± 3.8036.35\,\pm\,3.80 173.17± 16.93173.17\,\pm\,16.93 21.79± 2.8921.79\,\pm\,2.89 4.96± 1.664.96\,\pm\,1.66 0.32± 0.200.32\,\pm\,0.20 747.02± 90.35747.02\,\pm\,90.35
Pi_0.5 0.52± 0.13\mathbf{0.52}\,\pm\,0.13 0.41± 0.080.41\,\pm\,0.08 0.34± 0.030.34\,\pm\,0.03 0.07± 0.020.07\,\pm\,0.02 2.73± 0.622.73\,\pm\,0.62 3.86± 0.783.86\,\pm\,0.78 0.06± 0.100.06\,\pm\,0.10 14.23± 3.1314.23\,\pm\,3.13 27.71± 2.5427.71\,\pm\,2.54 139.02± 12.27139.02\,\pm\,12.27 9.53± 2.069.53\,\pm\,2.06 0.80± 0.670.80\,\pm\,0.67 0.28± 0.150.28\,\pm\,0.15 550.78± 117.31550.78\,\pm\,117.31
MolmoAct 0.32± 0.140.32\,\pm\,0.14 0.47± 0.070.47\,\pm\,0.07 0.28± 0.040.28\,\pm\,0.04 0.05± 0.01\mathbf{0.05}\,\pm\,0.01 2.21± 0.422.21\,\pm\,0.42 4.67± 0.834.67\,\pm\,0.83 0.44± 0.280.44\,\pm\,0.28 11.36± 1.8711.36\,\pm\,1.87 26.56± 4.3926.56\,\pm\,4.39 129.13± 20.69129.13\,\pm\,20.69 7.86± 1.257.86\,\pm\,1.25 0.48± 0.38\mathbf{0.48}\,\pm\,0.38 0.52± 0.280.52\,\pm\,0.28 646.72± 116.82646.72\,\pm\,116.82
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.49± 0.050.49\,\pm\,0.05 0.03± 0.010.03\,\pm\,0.01 0.61± 0.070.61\,\pm\,0.07 0.45± 0.060.45\,\pm\,0.06 0.00± 0.140.00\,\pm\,0.14 3.33± 0.343.33\,\pm\,0.34 62.59± 4.7062.59\,\pm\,4.70 287.53± 20.08287.53\,\pm\,20.08 2.40± 0.212.40\,\pm\,0.21 0.00± 0.140.00\,\pm\,0.14 0.00± 0.140.00\,\pm\,0.14 71.04± 9.6871.04\,\pm\,9.68
Lift Tray
Diffusion 0.00± 0.070.00\,\pm\,0.07 0.47± 0.060.47\,\pm\,0.06 0.47± 0.040.47\,\pm\,0.04 0.21± 0.040.21\,\pm\,0.04 5.01± 0.525.01\,\pm\,0.52 9.85± 0.289.85\,\pm\,0.28 5.28± 1.295.28\,\pm\,1.29 29.97± 2.2229.97\,\pm\,2.22 27.50± 3.4827.50\,\pm\,3.48 147.21± 13.54147.21\,\pm\,13.54 21.38± 1.7821.38\,\pm\,1.78 6.70± 2.206.70\,\pm\,2.20 0.48± 0.200.48\,\pm\,0.20 1000.00± 0.001000.00\,\pm\,0.00
GR00T 0.00± 0.070.00\,\pm\,0.07 0.46± 0.050.46\,\pm\,0.05 0.51± 0.020.51\,\pm\,0.02 0.12± 0.020.12\,\pm\,0.02 3.93± 0.303.93\,\pm\,0.30 13.28± 0.2513.28\,\pm\,0.25 6.62± 1.256.62\,\pm\,1.25 32.10± 1.1532.10\,\pm\,1.15 34.08± 1.9834.08\,\pm\,1.98 228.16± 6.87228.16\,\pm\,6.87 20.66± 0.9320.66\,\pm\,0.93 17.06± 2.2517.06\,\pm\,2.25 0.50± 0.230.50\,\pm\,0.23 1000.00± 0.001000.00\,\pm\,0.00
xVLA 0.00± 0.070.00\,\pm\,0.07 0.35± 0.040.35\,\pm\,0.04 0.63± 0.060.63\,\pm\,0.06 0.24± 0.040.24\,\pm\,0.04 7.01± 0.927.01\,\pm\,0.92 10.49± 0.4310.49\,\pm\,0.43 4.88± 0.944.88\,\pm\,0.94 37.46± 3.7237.46\,\pm\,3.72 32.82± 4.3932.82\,\pm\,4.39 151.68± 16.81151.68\,\pm\,16.81 28.46± 3.0928.46\,\pm\,3.09 10.46± 2.6810.46\,\pm\,2.68 0.16± 0.12\mathbf{0.16}\,\pm\,0.12 1000.00± 0.001000.00\,\pm\,0.00
Pi_0.5 0.32± 0.140.32\,\pm\,0.14 0.75± 0.070.75\,\pm\,0.07 0.52± 0.050.52\,\pm\,0.05 0.14± 0.030.14\,\pm\,0.03 4.83± 0.844.83\,\pm\,0.84 5.87± 0.945.87\,\pm\,0.94 2.62± 0.992.62\,\pm\,0.99 29.67± 5.0629.67\,\pm\,5.06 31.27± 2.3131.27\,\pm\,2.31 181.95± 11.42181.95\,\pm\,11.42 21.82± 3.7821.82\,\pm\,3.78 5.22± 2.075.22\,\pm\,2.07 0.92± 0.220.92\,\pm\,0.22 719.16± 115.13719.16\,\pm\,115.13
MolmoAct 0.46± 0.14\mathbf{0.46}\,\pm\,0.14 0.91± 0.05\mathbf{0.91}\,\pm\,0.05 0.38± 0.030.38\,\pm\,0.03 0.07± 0.020.07\,\pm\,0.02 2.18± 0.462.18\,\pm\,0.46 5.04± 1.06\mathbf{5.04}\,\pm\,1.06 0.98± 0.57\mathbf{0.98}\,\pm\,0.57 14.06± 2.9514.06\,\pm\,2.95 21.12± 2.4121.12\,\pm\,2.41 127.85± 13.57127.85\,\pm\,13.57 9.58± 2.049.58\,\pm\,2.04 1.98± 1.17\mathbf{1.98}\,\pm\,1.17 0.90± 0.350.90\,\pm\,0.35 589.02± 125.85\mathbf{589.02}\,\pm\,125.85
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.39± 0.040.39\,\pm\,0.04 0.03± 0.000.03\,\pm\,0.00 0.52± 0.030.52\,\pm\,0.03 0.56± 0.030.56\,\pm\,0.03 0.00± 0.060.00\,\pm\,0.06 2.87± 0.132.87\,\pm\,0.13 26.77± 2.9326.77\,\pm\,2.93 153.00± 16.82153.00\,\pm\,16.82 2.04± 0.082.04\,\pm\,0.08 0.00± 0.060.00\,\pm\,0.06 0.00± 0.060.00\,\pm\,0.06 83.62± 3.9683.62\,\pm\,3.96
Pack Box
Diffusion 0.04± 0.090.04\,\pm\,0.09 0.27± 0.060.27\,\pm\,0.06 0.43± 0.040.43\,\pm\,0.04 0.11± 0.020.11\,\pm\,0.02 7.19± 0.417.19\,\pm\,0.41 5.42± 0.315.42\,\pm\,0.31 0.58± 0.340.58\,\pm\,0.34 36.50± 2.5136.50\,\pm\,2.51 29.98± 2.0729.98\,\pm\,2.07 154.68± 11.16154.68\,\pm\,11.16 25.85± 1.8125.85\,\pm\,1.81 9.82± 3.059.82\,\pm\,3.05 0.44± 0.160.44\,\pm\,0.16 962.98± 44.27962.98\,\pm\,44.27
GR00T 0.00± 0.070.00\,\pm\,0.07 0.07± 0.030.07\,\pm\,0.03 0.50± 0.020.50\,\pm\,0.02 0.24± 0.040.24\,\pm\,0.04 4.77± 0.364.77\,\pm\,0.36 10.75± 0.2010.75\,\pm\,0.20 1.04± 0.471.04\,\pm\,0.47 29.29± 1.3729.29\,\pm\,1.37 40.49± 3.0840.49\,\pm\,3.08 207.72± 9.51207.72\,\pm\,9.51 19.50± 1.1019.50\,\pm\,1.10 13.30± 2.9213.30\,\pm\,2.92 0.30± 0.190.30\,\pm\,0.19 1000.00± 0.001000.00\,\pm\,0.00
xVLA 0.02± 0.080.02\,\pm\,0.08 0.12± 0.050.12\,\pm\,0.05 0.79± 0.050.79\,\pm\,0.05 0.25± 0.030.25\,\pm\,0.03 9.76± 0.889.76\,\pm\,0.88 7.29± 0.357.29\,\pm\,0.35 1.30± 0.531.30\,\pm\,0.53 47.57± 3.1047.57\,\pm\,3.10 46.60± 3.6346.60\,\pm\,3.63 200.34± 12.90200.34\,\pm\,12.90 34.53± 2.3634.53\,\pm\,2.36 9.24± 3.089.24\,\pm\,3.08 0.14± 0.11\mathbf{0.14}\,\pm\,0.11 988.12± 23.28988.12\,\pm\,23.28
Pi_0.5 0.54± 0.140.54\,\pm\,0.14 0.55± 0.060.55\,\pm\,0.06 0.73± 0.070.73\,\pm\,0.07 0.10± 0.010.10\,\pm\,0.01 4.60± 0.894.60\,\pm\,0.89 3.70± 0.723.70\,\pm\,0.72 0.90± 0.410.90\,\pm\,0.41 22.61± 4.5622.61\,\pm\,4.56 27.59± 1.3927.59\,\pm\,1.39 130.35± 7.35130.35\,\pm\,7.35 15.84± 3.2515.84\,\pm\,3.25 1.24± 1.191.24\,\pm\,1.19 0.44± 0.200.44\,\pm\,0.20 566.54± 113.98566.54\,\pm\,113.98
MolmoAct 0.62± 0.14\mathbf{0.62}\,\pm\,0.14 0.57± 0.05\mathbf{0.57}\,\pm\,0.05 0.58± 0.060.58\,\pm\,0.06 0.08± 0.020.08\,\pm\,0.02 3.22± 0.713.22\,\pm\,0.71 3.14± 0.78\mathbf{3.14}\,\pm\,0.78 0.44± 0.31\mathbf{0.44}\,\pm\,0.31 15.26± 3.7215.26\,\pm\,3.72 24.28± 1.5824.28\,\pm\,1.58 112.64± 7.83112.64\,\pm\,7.83 10.10± 2.5910.10\,\pm\,2.59 1.12± 0.721.12\,\pm\,0.72 0.48± 0.250.48\,\pm\,0.25 470.42± 115.06\mathbf{470.42}\,\pm\,115.06
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.51± 0.040.51\,\pm\,0.04 0.05± 0.000.05\,\pm\,0.00 1.30± 0.061.30\,\pm\,0.06 0.68± 0.040.68\,\pm\,0.04 0.84± 0.370.84\,\pm\,0.37 5.38± 0.345.38\,\pm\,0.34 28.24± 1.5028.24\,\pm\,1.50 134.77± 6.73134.77\,\pm\,6.73 3.35± 0.213.35\,\pm\,0.21 0.18± 0.160.18\,\pm\,0.16 0.03± 0.070.03\,\pm\,0.07 129.66± 8.00129.66\,\pm\,8.00
Pick Single Book From Table
Diffusion 0.02± 0.080.02\,\pm\,0.08 0.15± 0.090.15\,\pm\,0.09 0.59± 0.040.59\,\pm\,0.04 0.24± 0.030.24\,\pm\,0.03 4.42± 0.494.42\,\pm\,0.49 11.20± 0.8311.20\,\pm\,0.83 5.52± 1.275.52\,\pm\,1.27 28.17± 2.5928.17\,\pm\,2.59 21.24± 1.9821.24\,\pm\,1.98 136.43± 10.11136.43\,\pm\,10.11 17.38± 1.6917.38\,\pm\,1.69 4.26± 1.504.26\,\pm\,1.50 0.04± 0.090.04\,\pm\,0.09 926.04± 61.77926.04\,\pm\,61.77
GR00T 0.00± 0.070.00\,\pm\,0.07 0.05± 0.050.05\,\pm\,0.05 0.56± 0.030.56\,\pm\,0.03 0.26± 0.030.26\,\pm\,0.03 2.96± 0.362.96\,\pm\,0.36 15.47± 1.6315.47\,\pm\,1.63 9.12± 1.309.12\,\pm\,1.30 28.12± 2.6928.12\,\pm\,2.69 28.97± 1.9028.97\,\pm\,1.90 237.89± 7.47237.89\,\pm\,7.47 18.37± 1.9018.37\,\pm\,1.90 10.90± 2.4410.90\,\pm\,2.44 0.04± 0.090.04\,\pm\,0.09 821.42± 83.40821.42\,\pm\,83.40
xVLA 0.00± 0.070.00\,\pm\,0.07 0.02± 0.080.02\,\pm\,0.08 0.72± 0.060.72\,\pm\,0.06 0.21± 0.030.21\,\pm\,0.03 5.10± 0.725.10\,\pm\,0.72 13.53± 0.8113.53\,\pm\,0.81 5.58± 0.965.58\,\pm\,0.96 31.56± 2.9431.56\,\pm\,2.94 22.35± 2.7522.35\,\pm\,2.75 114.50± 11.31114.50\,\pm\,11.31 22.67± 2.3522.67\,\pm\,2.35 7.90± 1.757.90\,\pm\,1.75 0.02± 0.08\mathbf{0.02}\,\pm\,0.08 949.08± 47.67949.08\,\pm\,47.67
Pi_0.5 0.22± 0.130.22\,\pm\,0.13 0.44± 0.130.44\,\pm\,0.13 0.72± 0.070.72\,\pm\,0.07 0.20± 0.020.20\,\pm\,0.02 1.89± 0.471.89\,\pm\,0.47 6.03± 1.546.03\,\pm\,1.54 3.02± 0.883.02\,\pm\,0.88 11.37± 2.8811.37\,\pm\,2.88 17.94± 1.5117.94\,\pm\,1.51 91.87± 6.2391.87\,\pm\,6.23 6.67± 1.766.67\,\pm\,1.76 0.28± 0.240.28\,\pm\,0.24 0.12± 0.110.12\,\pm\,0.11 458.76± 120.93\mathbf{458.76}\,\pm\,120.93
MolmoAct 0.30± 0.14\mathbf{0.30}\,\pm\,0.14 0.48± 0.13\mathbf{0.48}\,\pm\,0.13 0.62± 0.080.62\,\pm\,0.08 0.18± 0.01\mathbf{0.18}\,\pm\,0.01 1.96± 0.351.96\,\pm\,0.35 8.79± 1.688.79\,\pm\,1.68 2.70± 0.772.70\,\pm\,0.77 13.06± 2.3813.06\,\pm\,2.38 15.55± 2.2215.55\,\pm\,2.22 82.43± 8.3182.43\,\pm\,8.31 7.10± 1.307.10\,\pm\,1.30 0.78± 0.590.78\,\pm\,0.59 0.02± 0.08\mathbf{0.02}\,\pm\,0.08 641.82± 123.19641.82\,\pm\,123.19
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.87± 0.000.87\,\pm\,0.00 0.18± 0.000.18\,\pm\,0.00 0.76± 0.000.76\,\pm\,0.00 1.20± 0.001.20\,\pm\,0.00 0.00± 0.000.00\,\pm\,0.00 4.09± 0.004.09\,\pm\,0.00 30.32± 0.0030.32\,\pm\,0.00 138.21± 0.00138.21\,\pm\,0.00 1.90± 0.001.90\,\pm\,0.00 0.00± 0.000.00\,\pm\,0.00 0.00± 0.000.00\,\pm\,0.00 118.00± 0.00118.00\,\pm\,0.00
Rotate Valve
Diffusion 0.04± 0.090.04\,\pm\,0.09 0.23± 0.070.23\,\pm\,0.07 0.44± 0.040.44\,\pm\,0.04 0.10± 0.020.10\,\pm\,0.02 2.10± 0.232.10\,\pm\,0.23 11.62± 0.5211.62\,\pm\,0.52 10.18± 1.4710.18\,\pm\,1.47 13.61± 1.1813.61\,\pm\,1.18 5.88± 0.545.88\,\pm\,0.54 41.95± 3.0841.95\,\pm\,3.08 9.76± 0.949.76\,\pm\,0.94 2.86± 1.062.86\,\pm\,1.06 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 977.16± 35.82977.16\,\pm\,35.82
GR00T 0.00± 0.070.00\,\pm\,0.07 0.11± 0.040.11\,\pm\,0.04 0.63± 0.030.63\,\pm\,0.03 0.08± 0.010.08\,\pm\,0.01 2.69± 0.172.69\,\pm\,0.17 18.11± 0.6318.11\,\pm\,0.63 23.70± 2.6423.70\,\pm\,2.64 21.46± 0.7721.46\,\pm\,0.77 14.40± 0.8114.40\,\pm\,0.81 88.73± 2.5088.73\,\pm\,2.50 15.09± 0.7515.09\,\pm\,0.75 16.40± 2.5016.40\,\pm\,2.50 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 1000.00± 0.001000.00\,\pm\,0.00
xVLA 0.10± 0.110.10\,\pm\,0.11 0.15± 0.060.15\,\pm\,0.06 0.73± 0.050.73\,\pm\,0.05 0.13± 0.020.13\,\pm\,0.02 3.07± 0.293.07\,\pm\,0.29 11.74± 0.6211.74\,\pm\,0.62 12.34± 2.1912.34\,\pm\,2.19 19.40± 1.3519.40\,\pm\,1.35 7.30± 0.557.30\,\pm\,0.55 38.85± 2.5338.85\,\pm\,2.53 14.54± 1.0614.54\,\pm\,1.06 2.40± 0.912.40\,\pm\,0.91 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 953.00± 44.60953.00\,\pm\,44.60
Pi_0.5 0.72± 0.14\mathbf{0.72}\,\pm\,0.14 0.88± 0.050.88\,\pm\,0.05 0.36± 0.020.36\,\pm\,0.02 0.03± 0.000.03\,\pm\,0.00 0.62± 0.140.62\,\pm\,0.14 8.65± 1.598.65\,\pm\,1.59 12.42± 2.3612.42\,\pm\,2.36 4.67± 0.954.67\,\pm\,0.95 2.99± 0.222.99\,\pm\,0.22 19.81± 1.2919.81\,\pm\,1.29 3.43± 0.693.43\,\pm\,0.69 0.02± 0.08\mathbf{0.02}\,\pm\,0.08 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 527.64± 95.25527.64\,\pm\,95.25
MolmoAct 0.72± 0.14\mathbf{0.72}\,\pm\,0.14 0.90± 0.05\mathbf{0.90}\,\pm\,0.05 0.30± 0.02\mathbf{0.30}\,\pm\,0.02 0.03± 0.00\mathbf{0.03}\,\pm\,0.00 0.45± 0.10\mathbf{0.45}\,\pm\,0.10 8.51± 1.96\mathbf{8.51}\,\pm\,1.96 8.84± 1.28\mathbf{8.84}\,\pm\,1.28 3.42± 0.73\mathbf{3.42}\,\pm\,0.73 2.56± 0.20\mathbf{2.56}\,\pm\,0.20 16.77± 1.05\mathbf{16.77}\,\pm\,1.05 2.48± 0.51\mathbf{2.48}\,\pm\,0.51 0.14± 0.240.14\,\pm\,0.24 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 481.94± 107.26\mathbf{481.94}\,\pm\,107.26
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.37± 0.040.37\,\pm\,0.04 0.01± 0.000.01\,\pm\,0.00 0.20± 0.020.20\,\pm\,0.02 1.98± 0.221.98\,\pm\,0.22 6.38± 0.736.38\,\pm\,0.73 1.24± 0.111.24\,\pm\,0.11 3.60± 0.313.60\,\pm\,0.31 22.26± 2.0622.26\,\pm\,2.06 1.04± 0.101.04\,\pm\,0.10 0.00± 0.190.00\,\pm\,0.19 0.00± 0.190.00\,\pm\,0.19 150.12± 16.61150.12\,\pm\,16.61
Stack Single Book Shelf
Diffusion 0.00± 0.070.00\,\pm\,0.07 0.01± 0.010.01\,\pm\,0.01 0.78± 0.060.78\,\pm\,0.06 0.24± 0.020.24\,\pm\,0.02 6.01± 0.636.01\,\pm\,0.63 11.95± 0.6211.95\,\pm\,0.62 5.74± 0.835.74\,\pm\,0.83 35.11± 2.5835.11\,\pm\,2.58 29.84± 3.9729.84\,\pm\,3.97 166.93± 15.36166.93\,\pm\,15.36 22.73± 1.8422.73\,\pm\,1.84 3.54± 1.223.54\,\pm\,1.22 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 966.84± 45.49966.84\,\pm\,45.49
GR00T 0.00± 0.070.00\,\pm\,0.07 0.04± 0.030.04\,\pm\,0.03 0.56± 0.020.56\,\pm\,0.02 0.29± 0.030.29\,\pm\,0.03 3.17± 0.413.17\,\pm\,0.41 16.17± 1.5416.17\,\pm\,1.54 9.46± 1.349.46\,\pm\,1.34 28.67± 2.4928.67\,\pm\,2.49 31.38± 2.5731.38\,\pm\,2.57 236.75± 5.66236.75\,\pm\,5.66 18.50± 1.7118.50\,\pm\,1.71 11.38± 2.7011.38\,\pm\,2.70 0.04± 0.090.04\,\pm\,0.09 842.26± 76.69842.26\,\pm\,76.69
xVLA 0.00± 0.070.00\,\pm\,0.07 0.01± 0.010.01\,\pm\,0.01 0.78± 0.060.78\,\pm\,0.06 0.24± 0.040.24\,\pm\,0.04 5.53± 0.775.53\,\pm\,0.77 13.64± 0.8213.64\,\pm\,0.82 7.28± 1.317.28\,\pm\,1.31 33.77± 3.2733.77\,\pm\,3.27 29.08± 3.5729.08\,\pm\,3.57 150.93± 15.57150.93\,\pm\,15.57 24.37± 2.4724.37\,\pm\,2.47 10.16± 2.4510.16\,\pm\,2.45 0.02± 0.080.02\,\pm\,0.08 937.06± 49.54937.06\,\pm\,49.54
Pi_0.5 0.12± 0.120.12\,\pm\,0.12 0.27± 0.100.27\,\pm\,0.10 0.79± 0.060.79\,\pm\,0.06 0.20± 0.010.20\,\pm\,0.01 2.04± 0.562.04\,\pm\,0.56 5.96± 1.60\mathbf{5.96}\,\pm\,1.60 2.16± 0.58\mathbf{2.16}\,\pm\,0.58 12.82± 3.5912.82\,\pm\,3.59 19.90± 1.2519.90\,\pm\,1.25 104.69± 5.90104.69\,\pm\,5.90 7.39± 2.097.39\,\pm\,2.09 2.16± 1.452.16\,\pm\,1.45 0.16± 0.170.16\,\pm\,0.17 414.04± 114.28\mathbf{414.04}\,\pm\,114.28
MolmoAct 0.18± 0.13\mathbf{0.18}\,\pm\,0.13 0.33± 0.11\mathbf{0.33}\,\pm\,0.11 0.67± 0.07\mathbf{0.67}\,\pm\,0.07 0.19± 0.02\mathbf{0.19}\,\pm\,0.02 2.27± 0.40\mathbf{2.27}\,\pm\,0.40 8.76± 1.698.76\,\pm\,1.69 2.80± 0.862.80\,\pm\,0.86 14.01± 2.59\mathbf{14.01}\,\pm\,2.59 16.60± 1.33\mathbf{16.60}\,\pm\,1.33 93.43± 6.99\mathbf{93.43}\,\pm\,6.99 7.45± 1.41\mathbf{7.45}\,\pm\,1.41 0.30± 0.31\mathbf{0.30}\,\pm\,0.31 0.10± 0.110.10\,\pm\,0.11 600.90± 117.81600.90\,\pm\,117.81
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.85± 0.000.85\,\pm\,0.00 0.20± 0.000.20\,\pm\,0.00 1.10± 0.001.10\,\pm\,0.00 2.12± 0.002.12\,\pm\,0.00 0.00± 0.000.00\,\pm\,0.00 6.29± 0.006.29\,\pm\,0.00 19.79± 0.0019.79\,\pm\,0.00 100.20± 0.00100.20\,\pm\,0.00 2.90± 0.002.90\,\pm\,0.00 0.00± 0.000.00\,\pm\,0.00 0.00± 0.000.00\,\pm\,0.00 194.00± 0.00194.00\,\pm\,0.00
Stack Two Blocks
Diffusion 0.06± 0.100.06\,\pm\,0.10 0.13± 0.050.13\,\pm\,0.05 0.34± 0.030.34\,\pm\,0.03 0.08± 0.010.08\,\pm\,0.01 4.57± 0.234.57\,\pm\,0.23 7.27± 0.247.27\,\pm\,0.24 4.58± 0.474.58\,\pm\,0.47 24.32± 1.5524.32\,\pm\,1.55 18.74± 0.8318.74\,\pm\,0.83 106.54± 4.95106.54\,\pm\,4.95 18.54± 1.2318.54\,\pm\,1.23 2.04± 0.662.04\,\pm\,0.66 0.10± 0.110.10\,\pm\,0.11 978.98± 29.57978.98\,\pm\,29.57
GR00T 0.04± 0.090.04\,\pm\,0.09 0.05± 0.030.05\,\pm\,0.03 0.50± 0.020.50\,\pm\,0.02 0.12± 0.020.12\,\pm\,0.02 4.70± 0.354.70\,\pm\,0.35 12.16± 0.6312.16\,\pm\,0.63 2.56± 0.502.56\,\pm\,0.50 32.41± 1.8532.41\,\pm\,1.85 41.87± 2.3141.87\,\pm\,2.31 233.11± 6.53233.11\,\pm\,6.53 22.17± 1.3922.17\,\pm\,1.39 23.60± 2.4923.60\,\pm\,2.49 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 965.66± 47.12965.66\,\pm\,47.12
xVLA 0.00± 0.070.00\,\pm\,0.07 0.01± 0.020.01\,\pm\,0.02 0.54± 0.050.54\,\pm\,0.05 0.17± 0.030.17\,\pm\,0.03 4.34± 0.464.34\,\pm\,0.46 8.74± 0.368.74\,\pm\,0.36 3.22± 0.593.22\,\pm\,0.59 27.38± 2.3227.38\,\pm\,2.32 22.73± 2.4322.73\,\pm\,2.43 115.21± 11.41115.21\,\pm\,11.41 19.44± 1.8619.44\,\pm\,1.86 9.34± 2.249.34\,\pm\,2.24 0.00± 0.07\mathbf{0.00}\,\pm\,0.07 974.08± 29.35974.08\,\pm\,29.35
Pi_0.5 0.40± 0.140.40\,\pm\,0.14 0.44± 0.070.44\,\pm\,0.07 0.34± 0.020.34\,\pm\,0.02 0.05± 0.010.05\,\pm\,0.01 2.03± 0.312.03\,\pm\,0.31 5.87± 0.885.87\,\pm\,0.88 2.16± 0.592.16\,\pm\,0.59 10.44± 1.7610.44\,\pm\,1.76 11.37± 0.7511.37\,\pm\,0.75 57.57± 4.1757.57\,\pm\,4.17 7.60± 1.297.60\,\pm\,1.29 1.94± 1.381.94\,\pm\,1.38 0.12± 0.120.12\,\pm\,0.12 710.78± 108.67710.78\,\pm\,108.67
MolmoAct 0.52± 0.13\mathbf{0.52}\,\pm\,0.13 0.48± 0.07\mathbf{0.48}\,\pm\,0.07 0.31± 0.04\mathbf{0.31}\,\pm\,0.04 0.04± 0.00\mathbf{0.04}\,\pm\,0.00 1.34± 0.29\mathbf{1.34}\,\pm\,0.29 4.70± 1.05\mathbf{4.70}\,\pm\,1.05 1.04± 0.36\mathbf{1.04}\,\pm\,0.36 6.16± 1.35\mathbf{6.16}\,\pm\,1.35 12.91± 1.46\mathbf{12.91}\,\pm\,1.46 58.79± 5.30\mathbf{58.79}\,\pm\,5.30 4.49± 1.02\mathbf{4.49}\,\pm\,1.02 0.38± 0.63\mathbf{0.38}\,\pm\,0.63 0.04± 0.090.04\,\pm\,0.09 545.88± 123.97\mathbf{545.88}\,\pm\,123.97
Human 1.00± 0.001.00\,\pm\,0.00 1.00± 0.001.00\,\pm\,0.00 0.40± 0.030.40\,\pm\,0.03 0.05± 0.000.05\,\pm\,0.00 0.42± 0.040.42\,\pm\,0.04 0.58± 0.060.58\,\pm\,0.06 0.24± 0.110.24\,\pm\,0.11 1.56± 0.171.56\,\pm\,0.17 23.24± 1.2823.24\,\pm\,1.28 95.97± 5.0495.97\,\pm\,5.04 1.11± 0.141.11\,\pm\,0.14 0.00± 0.050.00\,\pm\,0.05 0.00± 0.050.00\,\pm\,0.05 83.78± 9.0183.78\,\pm\,9.01

Abbreviations: Success = Success Rate [%]; TP = Task Progression [%]; BAVD = Bimanual Arm Velocity Difference [m/s]; BGVD = Bimanual Gripper Vertical Difference [m]; CPL = Cartesian Path Length [m]; CT = Completion Time [s]; ECC = Environment Collision Count; JPL = Joint Path Length [rad]; MCJ = Mean Cartesian Jerk [m/s3]; MJJ = Mean Joint Jerk [rad/s3]; OPL = Orientation Path Length [rad]; SCC = Self-Collision Count; SC = Slip Count; TL = Trajectory Length [steps].

C.2 Real-world Zero-shot DROID

The DROID hardware study uses a kitchen, three spatial positions and five trials per position, keeping camera positions the same across policies. Four tasks use binary completion counts; objects-in-bowl averages scores in {0,0.33,0.66,1}. Table 18 resolves why its percentages are not multiples of one-fifteenth. The prose’s π0 label differs from the π0.5-DROID table row.

Complete original PDF figure A3.F9
Figure 9: Sample trajectories from the DROID evaluation of MolmoAct2.
Complete original PDF table A3.T18
Table 18: Per-task and per-position success rates across policies. Each task uses 15 trajectories (5 per position).
Original selectable table
Task Policy Pos. 1 Pos. 2 Pos. 3 Overall
Put the apple on the plate MolmoAct2 5/5 (100%) 5/5 (100%) 5/5 (100%) 15/15 (100.0%)
π0​5\pi_{0}5-DROID 5/5 (100%) 3/5 (60%) 2/5 (40%) 10/15 (66.7%)
MolmoBot 5/5 (100%) 4/5 (80%) 4/5 (80%) 13/15 (86.7%)
Put the pipette in the tray MolmoAct2 4/5 (80%) 4/5 (80%) 5/5 (100%) 13/15 (86.7%)
π0​5\pi_{0}5-DROID 2/5 (40%) 3/5 (60%) 0/5 (0%) 5/15 (33.3%)
MolmoBot 3/5 (60%) 2/5 (40%) 3/5 (60%) 8/15 (53.3%)
Put the red cube inside the tape roll MolmoAct2 5/5 (100%) 4/5 (80%) 5/5 (100%) 14/15 (93.3%)
π0​5\pi_{0}5-DROID 3/5 (60%) 5/5 (100%) 0/5 (0%) 8/15 (53.3%)
MolmoBot 1/5 (20%) 0/5 (0%) 4/5 (80%) 5/15 (33.3%)
Put the knife in the box MolmoAct2 5/5 (100%) 5/5 (100%) 4/5 (80%) 14/15 (93.3%)
π0​5\pi_{0}5-DROID 3/5 (60%) 1/5 (20%) 0/5 (0%) 4/15 (26.7%)
MolmoBot 1/5 (20%) 0/5 (0%) 5/5 (100%) 6/15 (40.0%)
Put the objects in the bowla MolmoAct2 73.2% 53.0% 59.8% 62.0%
π0​5\pi_{0}5-DROID 39.6% 39.6% 59.4% 46.2%
MolmoBot 33.0% 26.4% 26.4% 28.6%
  • a

    Partial-credit task scored on {0, 0.33, 0.66, 1.0}\{0,\,0.33,\,0.66,\,1.0\} per trajectory; reported as mean success rate.

C.3 Real-world Bimanual YAM

Eight separate YAM task policies and the four baselines train under the stated matched protocol to convergence; Cortex AI collects and evaluates three spatial variants. Table 19 reports 50.6 versus 35.5 for OpenVLA on average, rather than the main text’s 50.1. Some entries are not binary multiples of 1/50, so the precise scoring or aggregation behind every fraction is not fully documented.

Complete original PDF figure A3.F10
Figure 10: Sample trajectories from the Bimanual YAM evaluation of MolmoAct2.
Complete original PDF figure A3.F11
Figure 11: Sample trajectories from the out-of-distribution Bimanual YAM evaluation of MolmoAct2.
Complete original PDF table A3.T19
Table 19: Overall success rate (%) across 8 manipulation tasks (50 episodes each). Best model per task is bold.
Original selectable table
Task π0.5\pi_{0.5} Cosmos X-VLA OpenVLA MolmoAct
Cup Stacking 62.0 64.5 5.5 60.5 54.0
Linearbot 11.6 9.2 5.2 24.4 64.0
Pegboard 2.0 0.0 3.0 4.5 14.0
Cup Storing 85.9 0.0 7.9 96.0 100.0
Candy Storing 26.0 0.0 3.8 26.5 40.5
Store Test Tube 17.0 11.0 0.0 42.0 67.0
Store Toys 26.0 7.0 2.5 2.0 32.0
Prepare Pipette 27.5 32.0 11.5 28.5 33.0
Average 32.2 15.5 4.9 35.5 50.6

C.4 Real-world SO-100/101 zero-shot

SO evaluation uses a fixed initialized camera and no further adaptation. Table 20 calls the comparison policy DePi, whereas the main table calls it π0-SO100/101; the matching scores are preserved without assuming identical training provenance beyond the source. Scores award 0.25 for reaching, 0.5 for pickup and 1 for placement, so 56.7 is weighted progress rather than full-placement success.

Complete original PDF table A3.T20
Table 20: Zero-shot success rate (%) on SO-100 tasks with fixed initial camera position. Each task is graded with partial credit at three milestones; partial scores of 0.25, 0.5, and 1.0 are awarded for reaching the corresponding stage. 15 episodes per task. Best model per task is bold.
Original selectable table
Partial-Credit Milestones Success Rate (%)
Task 0.25 0.5 1.0 SmolVLA MolmoAct v2 DePi0
Place fork on plate reach pickup fork place on plate 3.3 70.0 30.0
Stack blocks reach pickup block stack blocks 5.0 20.0 6.7
Place tissues in basket reach pickup place 0.0 73.3 20.0
Place pen on notebook reach pickup place 3.3 86.7 80.0
Place block in box reach pickup place 0.0 33.3 90.0
Average 2.3 56.7 45.3
Complete original PDF figure A3.F12
Figure 12: Sample trajectories from the SO-100 evaluation of MolmoAct2.

Appendix D Data Details

D.1 MolmoAct2-SO100/101 Dataset Statistics

TOPReward removes 438 of 1,660 dataset entries, leaving 1,222. The filtered corpus contains 921 SO-100, 299 SO-101 and two MOSS datasets, with six-dimensional state/action vectors and mostly 30 Hz, two-camera recordings. The FPS rows cover 1,218 datasets rather than the full 1,222, so four datasets are not accounted for in that breakdown.

Table 21 reports 12,818 globally unique instructions, while the annotation-comparison table lists 707 original SO instructions and 16,205 relabeled ones; the source does not reconcile their differing scopes. The verb plot demonstrates changed labeling distribution, not measured annotation accuracy.

Complete original PDF figure A4.F13
Figure 13: Action verb distribution before and after language annotation.
Complete original PDF figure A4.F14
Figure 14: Sample trajectories from the SO10x dataset partition.
Complete original PDF table A4.T21
Table 21: Summary statistics for the SO-100/101 training subset. We report statistics for the final filtered MolmoAct2-SO100/101 Dataset partition.
Original selectable table
Statistic Value
# datasets 1,222
Contributors 377
Episodes 38,059
Frames 19.8M
Estimated duration 183.6 hours
Metadata task count 1,322
Instruction rows 38,059
Globally unique instructions 12,818
Unique-instruction ratio 33.7%
Datasets with annotated task metadata 1,222
Complete original PDF table A4.T22
Table 22: SO-10x data partition breakdown by robot type.
Original selectable table
Robot type Datasets Episodes Frames Hours
SO-100 921 31,101 15.18M 141.7
SO-101 299 6,898 4.57M 41.6
MOSS 2 60 15.6K 0.2
Complete original PDF table A4.T23
Table 23: Recording and feature metadata. Most demonstrations are recorded at 30 Hz with two camera streams; all final filtered datasets use six-dimensional action and state vectors.
Original selectable table
Metadata field Value Datasets Episodes Frames
FPS 30 1,114 36,828 18.96M
FPS 20 78 527 261.4K
FPS 25 18 423 256.7K
FPS 60 8 208 231.6K
Camera count 1 230 4,159 2.26M
Camera count 2 835 24,151 13.05M
Camera count 3 151 9,235 4.11M
Camera count 4 6 514 342.1K
Action dimension 6 1,222 38,059 19.76M
State dimension 6 1,222 38,059 19.76M

D.2 Language Annotation Pipeline Implementation Details

Qwen3.5-27B runs at temperature 0.1 with at most 1,024 output tokens, using twelve uniformly spaced frames per non-frozen camera. Target instruction length is sampled between five and twenty-five words. The complete prompt asks for visible actions and verification of grasps; it is a generation instruction, not a guarantee that generated labels satisfy those requirements.

Table 24 reports increased unique-label counts across all datasets. Table 25 compares selected bimanual corpora and identifies YAM as largest among the listed open tabletop sets; its final row uses hours rather than trajectory count, so that row is not numerically comparable on the same axis.

Complete original PDF listing annotation-prompt
Original language-annotation prompt
Original selectable listing

You are an expert robot arm task instruction generator. You are provided with multiple demonstration videos, each showing the exact same task execution captured simultaneously from different camera angles. The task is to {original_instruction}. Based on these videos, generate a clear imperative instruction describing all of the actions performed by the robot arm. Carefully study the evolution of the state of the scene, to understand how the robot arm achieves the goal. Pay attention to the color and position of the objects the robot interacts with. Start directly with an action verb and make sure you do not miss any actions. Note that the robot arm is not always grasping an object and maybe pushing objects around instead. Verify any grasps by checking the gripper state, and only say the arm is grasping an object if this is shown in the wrist camera when the gripper is closed. Your prompts should be almost exactly {word_limit} words long.

Complete original PDF table A4.T24
Table 24: Dataset Language Annotation Diversity. Running the language annotation pipeline increases the number of unique instructions for all of the robotics datasets in MolmoAct2’s training mix.
Original selectable table
Dataset Original Instructions Relabeled Instructions
MolmoAct2-BimanualYAM Dataset 34 (0.1%) 11448 (35%)
MolmoAct2-DROID Dataset 49623 (67%) 55905 (75%)
MolmoAct2-SO100/101 Dataset 707 (1.5%) 16205 (34%)
MolmoAct Dataset 134 (1%) 2959 (28%)
BC-Z 104 (0.3%) 9981 (25%)
Fractal 598 (0.7%) 16729 (19%)
Bridge 19973 (37%) 35096 (66%)
Total 71121 (22%) 146485 (46%)
Complete original PDF table A4.T25
Table 25: Past Tabletop Bimanual Manipulation Datasets. MolmoAct2-BimanualYAM Dataset is the largest open-source tabletop bimanual robotics dataset. In this table, we compare against other open and closed-source tabletop bimanual datasets from the literature.
Original selectable table
Dataset Name Embodiment Number of Trajectories Open / Closed?
MolmoAct2-BimanualYAM Dataset (Ours) Bimanual YAM 34,500 Open
AIST Bimanual Manipulation Dataset (Motoda et al., 2025) ALOHA 10,000 Open
BiPlay Dataset (Dasari et al., 2025) ALOHA 7,000 Open
RDT-1B Finetuning Dataset (Liu et al., 2024) ALOHA 6,000 Open
ALOHA Unleashed Training Data (Zhao et al., 2024) ALOHA 26,000 Closed
SARM RA-BC Dataset (Chen et al., 2025) Bimanual YAM 200 Hours Closed

Appendix E Limitations and Potential Solutions

The explicit limitations focus on action-chunk execution and deployment coverage. The benchmark gains should be read together with these constraints, rather than as evidence of a universally reliable robot controller.

Action chunks and the absence of real-time re-chunking.

Each prediction spans one second and executes open-loop before another observation is used. Independent chunk training has no continuity loss across boundaries and can produce velocity or acceleration jumps. The policy cannot react within a chunk to contact, disturbances or tracking error; the reported 55.79 actions/s is amortized generation throughput.

Embodiment-specific zero-shot deployment.

Out-of-the-box use is tied to YAM, SO-100/101 and DROID Franka, where substantial embodiment-specific data is available. Other robots require demonstrations and adaptation. Broader coverage is a future data-scaling direction rather than a demonstrated property of the released checkpoints.

Concept demonstration

Open image ↗