Overview
The paper releases model checkpoints, training data and implementation code. The original resource links are collected below.
Models: Fine-tuned checkpoints · MolmoAct2 · Think · Pretrain · Molmo2-ER
Data: Bimanual YAM · SO-100/101 · DROID · Molmo2-ER
Code: allenai/molmoact2
Project: Release article
At a glance
MolmoAct2 combines an embodied-reasoning VLM, curated robot data, an open FAST action tokenizer and a continuous flow-matching expert conditioned on every VLM layer. Its Think variant caches depth tokens and regenerates changed regions. Results are strong on the reported benchmarks, but zero-shot deployment is embodiment-specific, hardware scores sometimes include partial credit, and one-second action chunks execute without within-chunk feedback.
1 Introduction
The project addresses three coupled obstacles: insufficient spatial grounding, the latency of explicit intermediate reasoning, and limited reproducibility of robot foundation models. It extends MolmoAct through a stronger backbone, robot data spanning three deployment platforms, transparent action tokenization, per-layer key/value conditioning and adaptive depth reasoning. Releasing these components supports inspection and adaptation; benchmark success does not establish dependable general-purpose deployment.
This edition covers the complete 51-page v2 of 8 May 2026, including all 14 figures, 25 tables, 23 numbered equations, both implementation listings and the full annotation prompt. The displayed author name Zhongzheng Ren follows this exact source version.
2 Molmo2-ER
Molmo2 is specialized for spatial and embodied tasks, then rehearsed on general multimodal data. Molmo2-ER reaches a reported 63.8 benchmark mean versus 46.8 for Molmo2. Its strongest result is the overall average; the source’s claim of beating every baseline on nine individual benchmarks overstates the values in Table 3, as detailed in the results notes.
Original selectable table
| #Samples | Weight | #Samples | Weight | ||
| Molmo2 Dataset | Molmo2-ER Dataset | ||||
| Image QA | 2.4M | 0.115 | Image Embodied QA | 1.3M | 0.11 |
| Video QA | 2.4M | 0.092 | Image Pointing | 780K | 0.11 |
| Image Pointing | 1.1M | 0.046 | Image Detection | 100K | 0.01 |
| Video Pointing | 370K | 0.069 | Video Embodied QA | 703k | 0.10 |
| Video Tracking | 800K | 0.069 | Multi-image/ Ego-Exo | 700K | 0.09 |
| Captions/Long QA | 1.2M | 0.069 | Abstract Reasoning | 150K | 0.04 |
| Subtotal | 8.3M | 0.46 | Subtotal | 3.3M | 0.46 |
| Tulu-3 SFT Dataset | 980k | 0.08 | Total | 12.5M | 1.0 |
2.1 Training data
The approximately 3.3-million-example embodied corpus combines six capability groups, from geometric image questions to video reasoning and cross-view correspondence. Supervision comes from simulation, annotated scans, templates and some generated reasoning traces. Table 1 reports sampling weights separately from raw sample counts; the rounded category sizes are not a consistent exact accounting of the stated subtotal.
Image embodied QA.
Image QA mixes SAT’s dynamic spatial reasoning, RoboPoint-QA’s general perception, RefSpatial’s referring and chain-of-thought examples, VST-P’s virtual-camera geometry and VSI-590K’s video-derived spatial supervision. The mixture spans real and synthetic data and multiple viewpoints rather than relying on one question template.
Video embodied QA.
SIMS-VSI contributes 203,000 simulator-grounded video questions about distance, direction, counting and order. A 200,000-example RoboVQA subset adds human-annotated long-horizon planning, affordances and prediction. These train perceptual reasoning, not directly measured robotic success.
Pointing and object detection.
Pixel localization uses 700,000 RoboPoint pointing examples, its 100,000 LVIS detection examples and RefSpatial’s 80,000 pointing examples. The normalized coordinate format grounds referring expressions and free-space descriptions in image locations.
Multi-image and ego–exo correspondence.
A 500,000-example SenseNova-SI subset and 200,000 cross-view VST-P examples train correspondence between cameras and between egocentric and external views. These complement the single-image data used elsewhere in the mixture.
Abstract embodied reasoning.
CLEVR contributes 50,000 compositional attribute/relation examples, while 100,000 GRiD-3D examples emphasize directions relative to an object’s own frame. This specifically separates object-relative language from camera-relative directions.
2.2 Specialize-then-rehearse training recipe
The specialize-then-rehearse recipe adapts a released checkpoint rather than repeating all original multimodal training. The second stage aims to recover general capability without discarding the new spatial specialization.
Stage 1: Embodied specialization.
Starting from the 4B Molmo2 mid-training checkpoint, specialization runs for 20,000 steps with 8% Tulu-3 text data. Sequence length is 4,200 and global batch size 64 on sixteen H100 GPUs. The text component is intended to retain language competence.
Stage 2: Joint refinement.
Joint refinement adds 1,500 steps, splitting the non-text budget equally between embodied and original Molmo2 data after testing embodied fractions 0.3, 0.5, 0.7 and 0.9. The resulting overall mixture is 46% embodied, 46% general multimodal and 8% text. Context grows to 16,384 tokens and device batch size falls to one.
3 Data
The robot mixture concentrates on bimanual YAM, community SO-100/101 and DROID Franka demonstrations, with smaller academic datasets and multimodal rehearsal. Collection and filtering improve practical coverage, but the mix remains heavily concentrated on its three intended deployment platforms.
3.1 MolmoAct2-BimanualYAM Dataset
The YAM release contains 34,500 teleoperated demonstrations, over 720 hours and more than 28 tasks, collected over two months with varied objects and layouts. Retry and idle-segment protocols are described as quality controls. Figure 3 shows a setup priced below US$6,000 at publication; that is hardware cost, not the cost of data collection or model training. Table 25 compares a selected set of tabletop bimanual datasets.
3.2 MolmoAct2-SO100/101 Dataset
SO data comes from 1,222 public LeRobot datasets and 377 contributors, totaling 38,059 episodes and roughly 184 hours. Filtering checks schema, corruption, evaluation-only collections and eligibility, then retains datasets whose mean TOPReward on their last three sampled episodes exceeds a human-audited reference threshold. This is a dataset-level proxy gate, not human verification of every episode.
3.3 MolmoAct2-DROID Dataset
The DROID subset uses successful episodes, extended instructions and idle-frame filtering from the supplementary annotations. It contains 74,604 episodes and 17,758,044 frames, with non-idle segments at least one second long. Instructions are re-annotated for the training mix, although the dedicated DROID fine-tune deliberately omits the additional annotations for comparison fairness.
3.4 Language annotation pipeline
Qwen3.5-27B sees sampled multi-camera frames and the original task description, then generates an imperative instruction with a randomized target word count. The reported global unique-label count rises from 71,121 to 146,485. More distinct strings are evidence of linguistic diversity, not an independent audit of semantic correctness; the complete annotation prompt is preserved in Appendix D.
3.5 Academic robotics datasets
BC-Z, BridgeData V2, RT-1 and the earlier MolmoAct dataset broaden camera, task and control conventions. They complement the three dominant robot datasets rather than supplying equal training mass.
3.6 Multimodal datasets
The non-robot portion is 46% Molmo2-ER, 46% original Molmo2 multimodal data and 8% Tulu-3 text. This internal composition is distinct from the overall VLA pretraining mixture, which samples 90% robot examples and 10% non-robot examples.
4 MolmoAct2
Training first creates a discrete robot-aware VLM, then attaches a continuous controller, then specializes to the deployment embodiment or task. Separating these stages lets the token interface learn before continuous expert alignment, and avoids requiring one expensive heterogeneous training run to finish every embodiment’s adaptation.
4.1 Pre-training
Discrete pretraining keeps the VLM’s image/text interface and adds state, setup, control and action tokens. It predicts one-second action chunks with the same next-token objective used for language. The continuous expert is introduced only afterward.
4.1.1 MolmoAct2-FAST Tokenizer
MolmoAct2-FAST follows FAST: a frequency transform, coefficient quantization and byte-pair encoding produce a 2,048-token action vocabulary. The contribution is an inspectable data mixture and released tokenizer rather than a different compression principle. Actions use percentile normalization, separate gripper handling and zero-padding to 32 dimensions.
The detailed recipe specifies one million sampled one-second sequences, despite the abstract’s “millions of trajectories” wording. Table 2 assigns 30% each to YAM, SO and DROID, with the remaining roughly 10% split among Fractal, BC-Z and Bridge. MolmoAct data is mentioned in prose but has no separate tokenizer-table row.
Original selectable table
| Dataset | Mix | Robot | Action representation |
| MolmoAct2-BimanualYAM Dataset | 30% | YAM | Absolute joint |
| MolmoAct2-SO100/101 Dataset | 30% | SO-100/101 | Absolute joint |
| MolmoAct2-DROID Dataset | 30% | Franka | Absolute joint |
| Fractal (Brohan et al., 2022) | 3.33% | Google Robot | Delta end-effector |
| BC-Z (Jang et al., 2022) | 3.33% | Google Robot | Delta end-effector |
| Bridge (Walke et al., 2023) | 3.33% | WidowX | Delta end-effector |
4.1.2 Pre-training recipe
Model.
The backbone retains SigLIP2 visual encoding, the Molmo2 connector and the 4B language model. Intermediate visual features are pooled and projected into language tokens. Appendix A supplies the architecture and the distinction between image and video pooling.
Data.
Pretraining samples 90% robot and 10% non-robot data; each major robot dataset receives 30% of the robot portion. Images use one resized crop, videos at most eight frames at up to 2 FPS, and camera order is randomized per episode. Setup/control descriptors identify the embodiment and action semantics, while geometric/color augmentation discourages dependence on a fixed visual arrangement.
Action and state representation.
Actions span one second at the dataset’s own rate, are normalized using first/99th-percentile statistics and padded to 32 dimensions before tokenization. State scalars are independently discretized into 256 bins. The shared vector width does not imply a shared physical coordinate system across joint-space and end-effector datasets.
Training.
Pretraining uses 200,000 steps, 4,200-token packed sequences and global batch 128 on 64 H100s, totaling about 5,760 GPU-hours. Attention masks separate packed examples. Learning rates are 5×10⁻⁶ for the vision encoder/connector and 10⁻⁵ for the language model; the new token embeddings also learn.
4.2 Post-training
Post-training adds a DiT-style continuous action expert while retaining discrete action supervision. The expert conditions on projected VLM keys and values at every layer. Ground-truth action-token positions are masked out of that context to prevent target leakage.
4.2.1 Action expert and per-layer KV conditioning
The flow model interpolates Gaussian noise toward the action chunk and regresses the constant target velocity, masking padded coordinates. Its 36 blocks each contain action self-attention, cross-attention to the corresponding VLM layer and a feed-forward branch, with time-dependent normalization and gates. Adapter projections map VLM keys/values into the expert width.
During post-training, detaching this conditioning path prevents the continuous loss from updating the VLM; the language-model loss still updates it. At inference, a learned velocity field is numerically integrated from noise, following the flow-matching formulation.
Equations 1–7
| (1) |
| (2) |
| (3) | ||||
| (4) | ||||
| (5) |
| (6) |
| (7) |
4.2.2 Post-training recipe
Overview.
The 200,000-step discrete checkpoint supplies the post-training initialization. Robot examples now carry both discrete action targets and continuous trajectories, while non-robot examples retain language-model supervision.
Multiple flow samples.
Four independent noise/time samples are drawn per action chunk and share the same VLM context. They provide multiple denoising targets for one demonstration without recomputing the backbone. Fine-tuning later uses eight samples; post-training stays at four because of GPU memory limits.
Equation 8
| (8) |
Padding and packing.
A shared tensor supports up to 30 time steps and 32 action dimensions. Masks exclude padded steps, coordinates and empty chunk rows from learning. Packed action chunks attend only to their corresponding prompt segment; a five-chunk padding cap improves shape stability when possible without discarding longer packed sequences.
Training objectives.
The objective adds language-model and flow losses. Discrete action targets supervise the backbone but are inaccessible to the expert’s conditioning context. Detached VLM keys/values provide knowledge insulation for the continuous objective while leaving ordinary autoregressive learning active.
Equation 9
| (9) |
Training.
Post-training runs 100,000 updates with global batch 128 on 64 H100s, about 2,304 GPU-hours. Robot sequences use 2,100 tokens, while non-robot batches retain 4,200. The new expert learns at 5×10⁻⁵; backbone learning rates remain unchanged.
4.3 Deployment
4.3.1 Embodiment-specific fine-tuning
Shared recipe.
Embodiment fine-tuning is robot-only, uses eight flow samples per chunk and allows continuous-loss gradients back into the VLM. It retains discrete action co-training, normalization, packing and the existing architecture. Added input-token embeddings are frozen in the main checkpoints while other specified parameters adapt; this is separate from the LoRA and expert-only ablations.
Bimanual YAM.
YAM uses fixed top/left/right cameras, annotated instructions, absolute joint targets and 30-step chunks at 30 Hz. Fine-tuning runs 100,000 updates with batch 128 on 64 H100s, about 2,304 GPU-hours.
DROID.
DROID uses an exterior camera followed by the wrist camera, absolute joint targets and 15-step chunks at 15 Hz. Alternate exterior views may be sampled during training. The dedicated run omits additional language annotations and uses 100,000 updates, batch 64 and 32 H100s, about 1,152 GPU-hours.
SO-100/101.
SO-100/101 randomizes camera order because community view layouts are inconsistent. It uses annotated instructions, absolute joint targets and 30-step chunks at 30 Hz, with 100,000 updates, batch 64 and 32 H100s.
LIBERO.
All four LIBERO suites train jointly with front/wrist cameras and relative end-effector actions. Ten-step chunks correspond to 10 Hz. Training runs 50,000 updates, with the best standard-model checkpoint selected at 40,000; Think selects a different checkpoint at 30,000.
Other evaluation fine-tunes.
Smaller downstream runs use the same robot-only structure, fixed metadata camera order and the dataset’s action rate/convention. The generic real-world recipe specifies 50,000 updates on eight H100s with batch 16 and best-checkpoint selection. Appendix C gives the distinct joint RoboEval recipe of 10,000 updates and batch 64.
4.3.2 Inference optimization
Within a chunk, the VLM context is fixed across ten flow steps. Caching context-dependent expert intermediates and fixed positional terms avoids repeated work; CUDA Graphs reduce launch overhead for the fixed-shape integration loop.
5 MolmoAct2-Think
Think predicts a 10×10 grid of depth codes before generating actions. Each cell uses one of 128 learned codes. Temporal reuse avoids regenerating the entire grid when only a small part of the scene changes, but depth generation still adds substantial latency relative to the action-only policy.
5.1 Adaptive depth perception data
Depth Anything V2 supplies monocular pseudo-depth, which the earlier MolmoAct VQ-VAE compresses from 320×320 pixels to 100 codes. RGB patches use cosine similarity with the previous frame; values below 0.996 trigger updates. Training targets the carried-forward depth buffer rather than independently refreshed depth at every cell.
This is an appearance-change heuristic, not a guarantee that unchanged RGB implies unchanged geometry. Slowly accumulating changes, camera motion and pseudo-depth errors can affect the representation.
Equation 10
| (10) |
5.2 Training
Post-training.
Depth post-training starts from the same discrete checkpoint and uniformly mixes action-only, depth-only and depth-plus-action examples. The joint style supplies depth tokens to the continuous expert while still masking target action tokens. It follows the standard post-training compute and optimization recipe with additional depth-token embeddings.
Fine-tuning.
LIBERO Think fine-tuning equally mixes action-only and depth-plus-action examples, corrupts 10% of teacher-forced depth inputs with random codes, and learns a per-layer gate on depth keys/values. The gate is derived from non-depth context and initialized with bias −4, initially limiting depth influence. The combined recipe addresses imperfect generated depth rather than assuming oracle depth at deployment.
Equations 11–12
| (11) |
| (12) |
5.3 Adaptive depth inference
Inference pipeline.
The first query generates all 100 codes; subsequent queries replay unchanged spans and greedily decode changed cells. Both replayed and new tokens enter the current VLM context before the action expert runs. Cached depth is therefore reused as token content, while the model still processes it to build the current context.
Inference optimization.
The adaptive scheduler remains eager because update patterns vary. A static cache and CUDA Graph stages accelerate fixed transformer work around attention; attention itself stays eager as its effective key/value length changes. This explains why graph replay benefits Think less than the regular action loop.
Original selectable table
| Models/Benchmarks |
Point-Bench
(Cheng et al., 2025)
|
RefSpatial
(Zhou et al., 2025)
|
RoboSpatial-Poi
(Song et al., 2025)
|
Where2Place
(Yuan et al., 2024)
|
CV-Bench
(Tong et al., 2024)
|
EmbSpatial
(Du et al., 2024)
|
MindCube
(Luo et al., 2026)
|
RoboSpatial-VQ
(Song et al., 2025)
|
OpenEQA
(Majumdar et al., 2024)
|
VSI-Bench
(Yang et al., 2025b)
|
Overall Avg. | |||
| API call only | ||||||||||||||
| GR-ER 1.5 Thinking (Team et al., 2025b) | 71.6 | 48.5 | 31.1 | 59.0 | 57.8 | 84.3 | 54.8 | 78.4 | 54.7 | 79.3 | 76.7 | 55.0 | 45.8 | 61.3 |
| GR-ER 1.5 (Team et al., 2025b) | 73.3 | 41.8 | 25.3 | 48.0 | 65.2 | 83.6 | 47.0 | 73.4 | 47.7 | 57.0 | 62.0 | 50.5 | 39.9 | 55.0 |
| Gemini 2.5 Pro (Comanici et al., 2025) | 62.7 | 33.6 | 8.3 | 37.0 | 69.2 | 85.9 | 56.0 | 78.0 | 59.2 | 71.3 | 74.7 | 55.0 | 51.1 | 57.1 |
| GPT-5 (Singh et al., 2025) | 43.6 | 23.5 | 19.0 | 37.0 | 71.3 | 86.1 | 59.0 | 81.5 | 58.0 | 69.3 | 86.7 | 64.4 | 52.9 | 57.9 |
| GPT-5-mini (Singh et al., 2025) | 39.5 | 23.0 | 12.5 | 33.5 | 56.4 | 85.9 | 57.3 | 78.8 | 55.6 | 70.7 | 81.3 | 59.2 | 46.2 | 53.8 |
| Open weights only | ||||||||||||||
| Qwen3-VL-4B (Yang et al., 2025a) | 63.8 | 49.5 | 62.3 | 63.0 | 65.2 | 85.8 | 39.5 | 78.1 | 27.4 | 62.7 | 60.7 | 47.6 | 61.4 | 59.0 |
| Qwen3-VL-8B (Yang et al., 2025a) | 64.2 | 47.5 | 56.6 | 63.0 | 66.6 | 85.6 | 41.5 | 78.2 | 33.4 | 73.7 | 70.0 | 49.3 | 63.3 | 61.0 |
| LLaVA-OV-7B (Li et al., 2024a) | 9.4 | 3.0 | 0.0 | 19.0 | 44.2 | 70.3 | 39.8 | 69.7 | 45.6 | 68.9 | 58.0 | 36.2 | 31.2 | 38.1 |
| InternVL3.5-4B (Wang et al., 2025a) | 43.0 | 21.5 | 23.0 | 53.0 | 58.7 | 80.5 | 37.0 | 72.0 | 42.6 | 61.0 | 49.3 | 44.6 | 56.5 | 49.4 |
| InternVL3.5-8B (Wang et al., 2025a) | 45.5 | 35.0 | 27.0 | 57.0 | 59.1 | 80.9 | 41.0 | 74.7 | 39.7 | 62.7 | 58.7 | 43.5 | 56.4 | 52.4 |
| Molmo family: Open weights, Open data, Open code | ||||||||||||||
| Molmo2 (Clark et al., 2026) | 76.9 | 52.5 | 13.9 | 18.0 | 50.8 | 64.4 | 46.3 | 67.0 | 37.6 | 57.9 | 50.0 | 46.9 | 26.1 | 46.8 |
| Molmo2-ER | 77.3 | 52.5 | 32.0 | 54.0 | 72.5 | 87.8 | 46.8 | 78.8 | 57.0 | 73.4 | 78.0 | 44.7 | 74.5 | 63.8 |
6 Experiments
Evaluation covers embodied VLM reasoning, embodiment-specific zero-shot transfer, simulation and hardware fine-tuning, perturbation robustness, trajectory diagnostics and speed. The tables use different outcome definitions and uncertainty conventions, so their percentages should be interpreted within each protocol rather than combined into one reliability claim.
Original selectable table
| Models/Tasks | Pick | Pick & Place | Open | Close | Average |
| StereoVLA (Deng et al., 2025) | 7.02.3 | – | – | – | 7.0 |
| LAP-VLA (Zha et al., 2026) | 24.92.7 | 6.61.6 | 11.41.5 | 45.92.9 | 22.2 |
| -DROID (Black et al., 2024) | 16.22.6 | 12.52.0 | 11.02.0 | 53.13.2 | 23.2 |
| -DROID (Intelligence et al., 2025) | 36.42.9 | 13.62.2 | 22.72.6 | 65.13.1 | 34.5 |
| MolmoAct2-DROID | 43.73.1 | 26.72.8 | 9.52.0 | 70.83.0 | 37.7 |
Original selectable table
| Models/Tasks | Pick MSProc | Pick Classic | Pick | Pick Rand.-Cam. | Pick&Place | PnP Next-To | PnP Color | Avg. |
| StereoVLA (Deng et al., 2025) | 6.62.6 | 4.31.5 | 1.11.0 | - | - | - | - | - |
| X-VLA (Zheng et al., 2025) | 3.31.0 | 0.50.5 | 0.70.5 | 0.80.5 | 0.10.2/0.10.2 | 1.90.9/1.00.7 | 0.90.6/0.50.5 | 1.2 |
| LAP-VLA (Zha et al., 2026) | 19.42.4 | 2.41.0 | 3.11.1 | 2.71.0 | 3.811.5/1.591.0 | 6.482.8/3.412.1 | 3.11.1/1.50.8 | 4.8 |
| -DROID (Intelligence et al., 2025) | 18.12.4 | 6.41.5 | 7.01.6 | 8.01.9 | 11.72.1/7.61.7 | 8.22.2/6.21.9 | 10.41.9/6.71.6 | 10.0 |
| MolmoAct2-DROID | 3.0 | 2.6 | 2.6 | 2.4 | 2.4/1.8 | 2.6/2.3 | 2.5/2.0 |
Original selectable table
| Models/Tasks | Apple on plate | Pipette in tray | Red cube in tape roll | Knife in box | Objects in bowl | Average |
| -DROID (Intelligence et al., 2025) | 66.7 | 33.3 | 53.3 | 26.7 | 46.2 | 45.2 |
| MolmoBot (Deshpande et al., 2026) | 86.7 | 53.3 | 33.3 | 40.0 | 28.6 | 48.4 |
| MolmoAct2-DROID | 100.0 | 86.7 | 93.3 | 93.3 | 62.0 | 87.1 |
Original selectable table
| Models/Tasks | Fork on plate | Stack blocks | Tissues in basket | Pen on notebook | Block in box | Average |
| SmolVLA (Shukor et al., 2025) | 3.3 | 5.0 | 0.0 | 3.3 | 0.0 | 2.3 |
| -SO100/101 (Chen, 2025) | 30.0 | 6.7 | 20.0 | 80.0 | 90.0 | 45.3 |
| MolmoAct2-SO100/101 | 70.0 | 20.0 | 73.3 | 86.7 | 33.3 | 56.7 |
Original selectable table
| Baseline | Spatial | Object | Goal | Long | Average |
| TraceVLA (Zheng et al., 2024) | 84.6% | 85.2% | 75.1% | 54.1% | 74.8% |
| OpenVLA (Kim et al., 2024) | 84.7% | 88.4% | 79.2% | 53.7% | 76.5% |
| SpatialVLA (Qu et al., 2025) | 88.2% | 89.9% | 78.6% | 55.5% | 78.1% |
| CoT-VLA (Zhao et al., 2025) | 87.5% | 91.6% | 87.6% | 69.0% | 83.9% |
| (Black et al., 2024) | 96.8% | 98.8% | 95.8% | 85.2% | 94.2% |
| ThinkAct (Huang et al., 2025) | 88.3% | 91.4% | 87.1% | 70.9% | 84.4% |
| MolmoAct-7B-D (Lee et al., 2025) | 87.0% | 95.4% | 87.6% | 77.2% | 86.6% |
| GR00T N1.7 (NVIDIA et al., 2025) | 97.7% | 97.5% | 98.5% | 94.4% | 97.0% |
| (Intelligence et al., 2025) | 98.8% | 98.2% | 98.0% | 92.4% | 96.9% |
| NORA-1.5 (Hung et al., 2025) | 97.3% | 96.4% | 94.5% | 89.6% | 94.5% |
| MolmoAct2 | 97.8% | 100.0% | 97.8% | 93.2% | 97.2% |
| MolmoAct2-Think | 98.8% | 99.8% | 98.5% | 95.4% | 98.1% |
6.1 Molmo2-ER evaluation
Evaluation setup.
Thirteen columns cover pointing, spatial VQA, cross-view reasoning and video understanding. RoboSpatial contributes separate pointing and VQA columns, explaining the difference between the number of named benchmark families and the table’s column count.
Baselines.
Comparisons include API-only Gemini/GPT models, open-weight Qwen/LLaVA/InternVL models and Molmo2. The table’s boldface marks open-weight leadership, not necessarily the best score across API and open models.
Results.
Molmo2-ER has the best reported mean, 63.8 versus 61.3 for Gemini-ER 1.5 Thinking and 46.8 for Molmo2: a 17-point improvement over its initialization. Table 3 gives four outright all-model wins and one RefSpatial tie. Nine leading results including that tie apply to the open-weight comparison, not nine outright wins over all models. It trails closed models on several embodied QA columns and even its initialization on OpenEQA.
6.2 Out-of-the-box deployment
The DROID checkpoint transfers without benchmark-specific retraining to DROID-like simulation and hardware. MolmoSpaces averages 37.7 versus 34.5 for π0.5-DROID, but Open is worse at 9.5 versus 22.7. MolmoBot averages 20.6 versus 10.0 using the table’s oracle pick-and-place successes; end-of-rollout successes are substantially lower, such as 7.8 versus 15.8 oracle on ordinary pick-and-place.
Hardware uses five tasks and fifteen trials each. DROID’s reported mean is 87.1 versus 48.4, but its objects-in-bowl task uses partial credit. SO’s 56.7 versus 45.3 is entirely a milestone-weighted mean, and block-in-box drops to 33.3 versus the baseline’s 90.0. Cameras are initially placed outside training layouts and then fixed across compared policies, not necessarily moved randomly every trial. These are gains on known embodiments, not universal zero-shot control. The source’s broad statistical-significance claim is not established for every comparison by trial counts alone.
6.3 Effective fine-tuning
Standard MolmoAct2 reaches 97.2 on LIBERO versus 97.0 for the strongest listed external baseline, and 44.3 on RoboEval versus about 40.5. Hardware adaptation trains separate single-task policies and evaluates eight YAM tasks with fifty episodes each; seven task rows favor MolmoAct2, while cup stacking favors another model.
The main text reports a 50.1 hardware mean, whereas Appendix Table 19 reports 50.6 and its rows average about 50.56. The task naming also differs: the appendix calls the 64-point row Linearbot while the main description names make_popcorn. Both original presentations are preserved rather than silently reconciled. These results demonstrate useful adaptation, but roughly half the evaluated hardware tasks still fail on average.
6.4 Performance of MolmoAct2-Think
Think improves LIBERO’s mean from 97.2 to 98.1, with Long rising from 93.2 to 95.4. Table 8 shows Object slipping from 100.0 to 99.8, despite the prose saying it matches; Goal is 98.5 there but 98.4 in Table 14. The gains are small and use best selected checkpoints. Two thousand evaluation rollouts without repeated training seeds or a reported paired significance analysis do not by themselves prove the improvement is non-incidental.
Original selectable table
| Model | Spatial Var. | Lighting | Language | Distractor | Overall |
| (Intelligence et al., 2025) | 15.00 | 33.70 | 26.15 | 33.20 | 27.01 |
| Cosmos Policy (Kim et al., 2026a) | 8.75 | 17.50 | 5.00 | 13.75 | 11.25 |
| X-VLA (Wen et al., 2025) | 3.75 | 8.30 | 9.95 | 3.75 | 6.44 |
| OpenVLA-OFT (Kim et al., 2025) | 13.75 | 46.25 | 51.25 | 48.30 | 39.89 |
| MolmoAct2-Think | 26.25 | 62.05 | 60.35 | 54.10 | 50.69 |
6.5 Robustness of MolmoAct2 to out-of-distribution shifts
Four hardware tasks are tested under spatial, lighting, instruction and distractor changes. The reported overall score is 50.69 versus 39.89 for OpenVLA-OFT, with the weakest category being spatial variation at 26.25. Table 9 labels the proposed row MolmoAct2-Think while the surrounding prose names MolmoAct2, leaving the exact variant ambiguous.
The stated schedule is twenty trials per task, five per perturbation; several fractional scores are not simple binary increments under that count, and the aggregation/partial-credit details are not fully specified. The result supports the reported comparison, not a precise uncertainty claim or general robustness guarantee.
6.6 Quality of MolmoAct2 trajectories for real-world deployment
RoboEval supplements success with path length, jerk, collisions, slipping and bimanual coordination. The paper highlights shorter completion times, including 4.70 seconds on Stack Two Blocks and 8.51 on Rotate Valve. The full combined-variation table reveals task-specific tradeoffs, so a favorable normalized radar plot should not be read as dominance on every raw metric or as a hardware safety certificate.
6.7 Systematic analysis
Ablations change the backbone, conditioning interface, number of flow samples, fine-tuning updates and depth regularization. Some start directly from the discrete checkpoint and therefore are not interchangeable with the final model’s full pretraining/post-training/fine-tuning pipeline.
Embodied-reasoning backbone.
With discrete actions only and 60,000 training steps on LIBERO Long, replacing Molmo2 by Molmo2-ER raises success from 77.6 to 83.6. This supports transfer of embodied specialization into action-token prediction under the matched setup.
Original selectable table
| Backbone | Action objective | Long |
| Molmo2 (Clark et al., 2026) | Discrete | 77.6% |
| Molmo2-ER | Discrete | 83.6% |
VLM-to-expert conditioning.
Conditioning on flattened per-layer keys/values averages 95.9, versus 94.8 for a head-preserving variant and 94.0 for hidden-state conditioning. The standard design is strongest on average, while hidden states remain best on Spatial. The result supports this tested interface rather than a universal claim about every VLM/expert architecture.
Multiple flow samples.
Increasing flow samples from one to eight improves the four-suite mean from 94.15 to 95.90. Per-suite gains are not monotonic: two samples score 92.2 on Long versus 89.8 with eight. Additional samples also add expert computation.
Original selectable table
| Spatial | Object | Goal | Long | Average | |
| 1 | 95.0% | 95.8% | 98.2% | 87.6% | 94.15% |
| 2 | 96.6% | 96.0% | 95.4% | 92.2% | 95.05% |
| 4 | 96.2% | 98.4% | 97.0% | 89.0% | 95.15% |
| 8 | 96.2% | 99.0% | 98.6% | 89.8% | 95.90% |
Fine-tuning design.
The final full fine-tune averages 97.20, versus 96.95 without discrete co-training, 97.05 with knowledge insulation, 96.25 with LoRA and 93.05 when only the expert learns. The small differences among full-model variants lack repeated-seed uncertainty, while the expert-only gap is larger. Removing co-training improves Long despite lowering the overall mean.
Original selectable table
| Discrete co-training | Knowledge insulation | Training type | Spatial | Object | Goal | Long | Average |
| ✗ | ✗ | Full fine-tuning | 95.8% | 98.6% | 98.4% | 95.0% | 96.95% |
| ✓ | ✓ | Full fine-tuning | 96.8% | 99.4% | 97.2% | 94.8% | 97.05% |
| ✓ | ✗ | LoRA | 98.0% | 99.4% | 97.2% | 90.4% | 96.25% |
| ✗ | ✗ | Action expert only | 91.6% | 97.6% | 93.8% | 89.2% | 93.05% |
| ✓ | ✗ | Full fine-tuning | 97.8% | 100.0% | 97.8% | 93.2% | 97.20% |
Depth-aware fine-tuning.
The combined depth-noise/gate/mixed-training recipe gives 98.10, versus 97.65 without noise and gating and 97.50 without all three. Noise and gating are removed together, so this table cannot isolate their individual contributions. Long is actually highest in the unregularized depth-only row.
Original selectable table
| Mixed training | Noise injection | Depth gate | Spatial | Object | Goal | Long | Average |
| ✗ | ✗ | ✗ | 97.6% | 99.6% | 96.6% | 96.2% | 97.50% |
| ✓ | ✗ | ✗ | 98.8% | 99.8% | 96.6% | 95.4% | 97.65% |
| ✓ | ✓ | ✓ | 98.8% | 99.8% | 98.4% | 95.4% | 98.10% |
6.8 Inference speed
On one H100 with horizon ten, caching plus CUDA Graphs raises action-only throughput from 23.02 to 55.79 actions/s and Think from 8.04 to 12.71. These correspond to roughly 179 ms and 787 ms per generated chunk, not observation-feedback frequencies of 55.79 and 12.71 Hz. The optimized discrete paths are slower at 14.17 and 6.82 actions/s.
Appendix E states that full one-second chunks execute open-loop before re-querying. No within-chunk correction or boundary-continuity objective is provided, so high generation throughput should not be confused with rapid closed-loop reaction.
7 Related work
Generalist robot manipulation policies.
Generalist policies transfer visual/language representations into robot imitation learning across datasets and embodiments. This work emphasizes releasing training ingredients as well as weights, while its demonstrated deployment coverage remains concentrated on the trained platforms.
Embodied reasoning for robotic manipulation.
Embodied reasoning may use points, boxes, future images, trajectories or latent states. Adaptive depth extends MolmoAct by reusing stable scene regions. The representation is interpretable as predicted geometry, but no causal faithfulness test establishes that each displayed depth token explains the resulting action.
Bridging VLM and action expert.
The architecture joins discrete autoregressive training with a continuous expert. Its distinguishing interface exposes corresponding-layer keys/values instead of only final hidden states. Depth reasoning changes the expert’s context, while flow matching produces the entire continuous chunk.
8 Conclusion
The contribution is a reproducible family of robot models and data with strong reported transfer and fine-tuning results. Practical limits remain substantial: embodiment-specific data dependence, low absolute success on some tasks, partial-credit metrics, expensive training and open-loop chunk execution. The complete source artifacts make those strengths and limits inspectable.
Author contributions
Haoquan Fang and Jiafei Duan share first authorship. The contribution statement credits specialist leads for tokenizer, data curation, backbone, depth reasoning and inference, along with hardware, evaluation, infrastructure and advisory teams. Ranjay Krishna is identified as project PI.
Acknowledgment
The authors acknowledge Ai2 colleagues and cluster support, Cirrascale infrastructure and Jiafei Duan’s A*STAR National Science Fellowship.
Appendix
The appendices supply architecture, training implementation, evaluation protocols, data statistics and explicit deployment limitations. All implementation listings, prompts and supplementary visual results are retained.
Appendix A Model Details
The model first learns a discrete robot interface, then gains a continuous expert. Think adds a depth prefix without replacing the backbone/expert structure. The following details specify the released architecture rather than a separate experimental system.
A.1 MolmoAct2 Backbone
Backbone initialization and visual inputs.
Robot observations use a single resized crop per camera to control sequence length. Video input is capped at eight frames and 2 FPS; camera serialization is randomized during pretraining. This trades fine visual detail for manageable multi-view context.
Vision encoder and connector.
SigLIP2 features from the third-to-last and ninth-to-last layers enter attention pooling: 2×2 windows for images and 3×3 for videos. A shared MLP maps pooled features into the language-model embedding space.
Language model input.
Image-index text and frame markers identify visual inputs. Visual tokens can attend across views and frames before action generation. The robot pipeline uses single crops rather than the base model’s tiled-image column machinery.
Added tokens.
Setup/control/state delimiters and output triggers distinguish robot semantics from task language. The vocabulary adds 2,048 action tokens and 256 state tokens; Think adds 128 depth codes plus its trigger/delimiters. The appendix’s illustrative DROID delta-end-effector prompt should not override the dedicated DROID checkpoint’s stated absolute-joint control.
Table 15 lists a 380M vision encoder, 57M connector, 4B language model and 621M action expert. The expert width is 768 with 36 layers and eight attention heads; architecture size is larger than the language-model-only 4B label.
Original selectable table
| Hyperparameter | Image Encoder | V/L Connector | LLM | Action Expert |
| Params | 380M | 57M | 4.0B | 621M |
| Dim | 1152 | 1152 | 2560 | 768 |
| MLP Dim | 4304 | 9728 | 9728 | 3072 |
| Act. | GELU | SwiGLU | SwiGLU | SwiGLU |
| Heads | 16 | 16 | 32 | 8 |
| KV Heads | 16 | – | 8 | – |
| Layers | 27 | – | 36 | 36 |
| Dropout | 0.0 | 0.0 | 0.1 | 0.0 |
| Image Size | – | – | – | |
| Patch Size | 14 | – | – | – |
| Image Pool Size | – | – | – | |
| Video Pool Size | – | – | – | |
| Pool Dim | – | 1152 | – | – |
| Pool Heads | – | 16 | – | – |
| Embed | – | – | 151936 | – |
| Theta | – | – | 1M | – |
| Max Horizon | – | – | – | 30 |
| Max Action Dim | – | – | – | 32 |
| Time Embed Dim | – | – | – | 256 |
| Conditioning | – | – | – | Per-layer KV |
A.2 Continuous Action Expert
Input and output.
The expert is a noncausal transformer over the noisy horizon-by-32 action tensor. It uses horizon thirty in post-training and ten for LIBERO, with padding masks removing unused coordinates and time steps.
Flow-matching objective.
The exact flow equations retain noise-to-data interpolation and masked squared-error training. Released inference uses ten Euler steps, then slices the output to the embodiment’s action width and reverses its normalization. Numerical integration and imitation learning provide no explicit action-safety constraint.
Equations 13–15
| (13) |
| (14) |
| (15) |
Expert block.
Each of 36 blocks combines bidirectional action attention, visual-language cross-attention and an MLP. Time embeddings modulate RMS normalization and residual gates, query/key normalization stabilizes attention, and rotary embeddings order the action sequence. The output projection begins near zero velocity.
Equations 16–18
| (16) | ||||
| (17) | ||||
| (18) |
Per-layer KV conditioning.
The VLM supplies eight 128-wide key/value heads, flattened to 1,024 dimensions per token. Learned adapters map them to the expert’s 768 dimensions, arranged as eight 96-wide heads. Detachment during post-training applies to this continuous-loss path, not to all backbone learning.
Equations 19–20
| (19) |
| (20) |
A.3 Adaptive-Depth Extension
Depth tokens form a 100-position prefix generated before actions. A learned sigmoid gate, computed from non-depth context, scales depth keys and values separately at each layer. Bias −4 initially suppresses their influence, and fine-tuning learns how much depth context to use.
Equations 21–22
| (21) |
| (22) |
Appendix B Training Details
B.1 Implementation
Training infrastructure.
Training uses PyTorch FSDP2, compiled static shapes and scaled-dot-product attention with custom masks. Most operations use bfloat16; normalization and rotary-position calculations retain full precision. Token losses normalize using the mean target-token count across devices, avoiding disproportionate weighting of short targets. The appendix says fewer than 0.1% of overlong examples are truncated, whereas the main recipe describes skipping them.
Packing.
Each loader worker pools 48 examples and solves a token/image packing problem with image weight thirty, rounding token counts to multiples of 32. Masks preserve independence among packed examples. Generic Molmo2 sequence limits listed here should not be confused with MolmoAct2’s stage-specific 4,200/2,100-token budgets.
Equation 23
| (23) |
Image augmentation.
The exact augmentation listing applies a 95% crop, resize, ±5-degree rotation, color jitter and Gaussian blur with probability 0.2. It applies to training images/videos and is disabled at evaluation. The source code is preserved alongside its native PDF rendering.
Original selectable listing
Input prompts.
The three exact templates select action, depth or depth-then-action output using assistant-side triggers. Task text is normalized, while setup, control mode and optional state tokens occupy distinct fields. The same chat prefix is used at training and inference. All delimiters and example token sequences are retained.
Original selectable listing
Training hyperparameters.
Table 16 records stage hyperparameters, including 200 warm-up steps, optimizer ε=10⁻⁶ and β=(0.9,0.95), learning rates and compute. Its pretraining action-expert entries are template values: the method explicitly introduces that expert only at post-training. Fine-tuning figures describe major embodiment runs; LIBERO and smaller evaluations have separate schedules.
Original selectable table
| Hyperparameter | Pre-train | Post-train | Fine-tune |
| Warm-up ViT | 200 | 200 | 200 |
| Warm-up Conn. | 200 | 200 | 200 |
| Warm-up LLM | 200 | 200 | 200 |
| Warm-up AE | 200 | 200 | 200 |
| LR ViT | |||
| LR Conn. | |||
| LR LLM | |||
| LR AE | |||
| Cosine Decay | 10% | 10% | 10% |
| Eps. | |||
| Betas | 0.9/0.95 | 0.9/0.95 | 0.9/0.95 |
| Steps | 200k | 100k | 100k |
| Global Batch Size | 128 | 128 | 64 or 128 (BimanualYAM) |
| Sequence Length | 4200 | 2100 (Robot) and 4200 (Multimodal) | 2100 |
| GPUs (H100s) | 64 | 64 | 32 or 64 (BimanualYAM) |
| Time (Hours) | 90 | 36 | 36 |
| GPU Hours | 5760 | 2304 | 1152 or 2304 (BimanualYAM) |
B.2 MolmoAct2-FAST Tokenizer
The tokenizer keeps FAST’s DCT/BPE compression logic and exposes its training mixture. Its openness makes the action vocabulary reproducible; it does not create a coordinate-invariant controller by itself.
Data Format
Single-arm vectors place arm coordinates and a gripper value before zero padding. Bimanual vectors concatenate left arm/gripper then right arm/gripper. The shared maximum is 32 dimensions.
Multimodal Representation
The tokenizer supports multiple control conventions rather than transforming all actions into one coordinate frame. Setup/control descriptors and visual context let the backbone distinguish absolute joint targets from end-effector commands.
B.3 GPU Cluster
Runs use Ai2’s Jupiter cluster, scheduled through Beaker and operated by Cirrascale. The cluster description gives infrastructure capacity, not the number of GPUs used by every experiment.
Compute
Jupiter has 1,024 H100 80GB GPUs across 128 servers, with dual Xeon CPUs, 2TB RAM and 18TB local NVMe per server. Individual training runs use the smaller GPU counts stated in their recipes.
Storage
Shared WEKA storage provides roughly 1PB NVMe plus 5PB HDD. The reported 761Gbps aggregate benchmark uses 64 clients; it is a cluster storage measurement rather than model training throughput.
Interconnect
A balanced rail-optimized InfiniBand network supports cross-node communication with eight 400Gbps adapters per server. This provides the infrastructure for multi-node training.
Cooling
Water-cooled closed-loop cabinets are reported to achieve datacenter PUE 1.2, with typical H100 temperatures of 60–65°C. These operational figures are not a measured energy or carbon accounting for this training project.
Appendix C Evaluation Details
C.1 RoboEval
RoboEval fine-tunes all eight tasks jointly for 10,000 steps on eight GPUs with batch 64. Evaluation varies position, orientation and task-specific conditions, and logs success, progress, path/jerk, contact and coordination metrics. Table 17 includes the full combined-variation results and human statistics from successful demonstrations only; that human row is not an identically sampled policy evaluation.
Original selectable table
| Method | Success | TP | BAVD | BGVD | CPL | CT | ECC | JPL | MCJ | MJJ | OPL | SCC | SC | TL |
| Cube Handover | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Lift Pot | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Lift Tray | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Pack Box | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Pick Single Book From Table | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Rotate Valve | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Stack Single Book Shelf | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human | ||||||||||||||
| Stack Two Blocks | ||||||||||||||
| Diffusion | ||||||||||||||
| GR00T | ||||||||||||||
| xVLA | ||||||||||||||
| Pi_0.5 | ||||||||||||||
| MolmoAct | ||||||||||||||
| Human |
Abbreviations: Success = Success Rate [%]; TP = Task Progression [%]; BAVD = Bimanual Arm Velocity Difference [m/s]; BGVD = Bimanual Gripper Vertical Difference [m]; CPL = Cartesian Path Length [m]; CT = Completion Time [s]; ECC = Environment Collision Count; JPL = Joint Path Length [rad]; MCJ = Mean Cartesian Jerk [m/s3]; MJJ = Mean Joint Jerk [rad/s3]; OPL = Orientation Path Length [rad]; SCC = Self-Collision Count; SC = Slip Count; TL = Trajectory Length [steps].
C.2 Real-world Zero-shot DROID
The DROID hardware study uses a kitchen, three spatial positions and five trials per position, keeping camera positions the same across policies. Four tasks use binary completion counts; objects-in-bowl averages scores in {0,0.33,0.66,1}. Table 18 resolves why its percentages are not multiples of one-fifteenth. The prose’s π0 label differs from the π0.5-DROID table row.
Original selectable table
| Task | Policy | Pos. 1 | Pos. 2 | Pos. 3 | Overall |
| Put the apple on the plate | MolmoAct2 | 5/5 (100%) | 5/5 (100%) | 5/5 (100%) | 15/15 (100.0%) |
| -DROID | 5/5 (100%) | 3/5 (60%) | 2/5 (40%) | 10/15 (66.7%) | |
| MolmoBot | 5/5 (100%) | 4/5 (80%) | 4/5 (80%) | 13/15 (86.7%) | |
| Put the pipette in the tray | MolmoAct2 | 4/5 (80%) | 4/5 (80%) | 5/5 (100%) | 13/15 (86.7%) |
| -DROID | 2/5 (40%) | 3/5 (60%) | 0/5 (0%) | 5/15 (33.3%) | |
| MolmoBot | 3/5 (60%) | 2/5 (40%) | 3/5 (60%) | 8/15 (53.3%) | |
| Put the red cube inside the tape roll | MolmoAct2 | 5/5 (100%) | 4/5 (80%) | 5/5 (100%) | 14/15 (93.3%) |
| -DROID | 3/5 (60%) | 5/5 (100%) | 0/5 (0%) | 8/15 (53.3%) | |
| MolmoBot | 1/5 (20%) | 0/5 (0%) | 4/5 (80%) | 5/15 (33.3%) | |
| Put the knife in the box | MolmoAct2 | 5/5 (100%) | 5/5 (100%) | 4/5 (80%) | 14/15 (93.3%) |
| -DROID | 3/5 (60%) | 1/5 (20%) | 0/5 (0%) | 4/15 (26.7%) | |
| MolmoBot | 1/5 (20%) | 0/5 (0%) | 5/5 (100%) | 6/15 (40.0%) | |
| Put the objects in the bowla | MolmoAct2 | 73.2% | 53.0% | 59.8% | 62.0% |
| -DROID | 39.6% | 39.6% | 59.4% | 46.2% | |
| MolmoBot | 33.0% | 26.4% | 26.4% | 28.6% |
- a
Partial-credit task scored on per trajectory; reported as mean success rate.
C.3 Real-world Bimanual YAM
Eight separate YAM task policies and the four baselines train under the stated matched protocol to convergence; Cortex AI collects and evaluates three spatial variants. Table 19 reports 50.6 versus 35.5 for OpenVLA on average, rather than the main text’s 50.1. Some entries are not binary multiples of 1/50, so the precise scoring or aggregation behind every fraction is not fully documented.
Original selectable table
| Task | Cosmos | X-VLA | OpenVLA | MolmoAct | |
| Cup Stacking | 62.0 | 64.5 | 5.5 | 60.5 | 54.0 |
| Linearbot | 11.6 | 9.2 | 5.2 | 24.4 | 64.0 |
| Pegboard | 2.0 | 0.0 | 3.0 | 4.5 | 14.0 |
| Cup Storing | 85.9 | 0.0 | 7.9 | 96.0 | 100.0 |
| Candy Storing | 26.0 | 0.0 | 3.8 | 26.5 | 40.5 |
| Store Test Tube | 17.0 | 11.0 | 0.0 | 42.0 | 67.0 |
| Store Toys | 26.0 | 7.0 | 2.5 | 2.0 | 32.0 |
| Prepare Pipette | 27.5 | 32.0 | 11.5 | 28.5 | 33.0 |
| Average | 32.2 | 15.5 | 4.9 | 35.5 | 50.6 |
C.4 Real-world SO-100/101 zero-shot
SO evaluation uses a fixed initialized camera and no further adaptation. Table 20 calls the comparison policy DePi, whereas the main table calls it π0-SO100/101; the matching scores are preserved without assuming identical training provenance beyond the source. Scores award 0.25 for reaching, 0.5 for pickup and 1 for placement, so 56.7 is weighted progress rather than full-placement success.
Original selectable table
| Partial-Credit Milestones | Success Rate (%) | |||||
| Task | 0.25 | 0.5 | 1.0 | SmolVLA | MolmoAct v2 | DePi0 |
| Place fork on plate | reach | pickup fork | place on plate | 3.3 | 70.0 | 30.0 |
| Stack blocks | reach | pickup block | stack blocks | 5.0 | 20.0 | 6.7 |
| Place tissues in basket | reach | pickup | place | 0.0 | 73.3 | 20.0 |
| Place pen on notebook | reach | pickup | place | 3.3 | 86.7 | 80.0 |
| Place block in box | reach | pickup | place | 0.0 | 33.3 | 90.0 |
| Average | 2.3 | 56.7 | 45.3 | |||
Appendix D Data Details
D.1 MolmoAct2-SO100/101 Dataset Statistics
TOPReward removes 438 of 1,660 dataset entries, leaving 1,222. The filtered corpus contains 921 SO-100, 299 SO-101 and two MOSS datasets, with six-dimensional state/action vectors and mostly 30 Hz, two-camera recordings. The FPS rows cover 1,218 datasets rather than the full 1,222, so four datasets are not accounted for in that breakdown.
Table 21 reports 12,818 globally unique instructions, while the annotation-comparison table lists 707 original SO instructions and 16,205 relabeled ones; the source does not reconcile their differing scopes. The verb plot demonstrates changed labeling distribution, not measured annotation accuracy.
Original selectable table
| Statistic | Value |
| # datasets | 1,222 |
| Contributors | 377 |
| Episodes | 38,059 |
| Frames | 19.8M |
| Estimated duration | 183.6 hours |
| Metadata task count | 1,322 |
| Instruction rows | 38,059 |
| Globally unique instructions | 12,818 |
| Unique-instruction ratio | 33.7% |
| Datasets with annotated task metadata | 1,222 |
Original selectable table
| Robot type | Datasets | Episodes | Frames | Hours |
| SO-100 | 921 | 31,101 | 15.18M | 141.7 |
| SO-101 | 299 | 6,898 | 4.57M | 41.6 |
| MOSS | 2 | 60 | 15.6K | 0.2 |
Original selectable table
| Metadata field | Value | Datasets | Episodes | Frames |
| FPS | 30 | 1,114 | 36,828 | 18.96M |
| FPS | 20 | 78 | 527 | 261.4K |
| FPS | 25 | 18 | 423 | 256.7K |
| FPS | 60 | 8 | 208 | 231.6K |
| Camera count | 1 | 230 | 4,159 | 2.26M |
| Camera count | 2 | 835 | 24,151 | 13.05M |
| Camera count | 3 | 151 | 9,235 | 4.11M |
| Camera count | 4 | 6 | 514 | 342.1K |
| Action dimension | 6 | 1,222 | 38,059 | 19.76M |
| State dimension | 6 | 1,222 | 38,059 | 19.76M |
D.2 Language Annotation Pipeline Implementation Details
Qwen3.5-27B runs at temperature 0.1 with at most 1,024 output tokens, using twelve uniformly spaced frames per non-frozen camera. Target instruction length is sampled between five and twenty-five words. The complete prompt asks for visible actions and verification of grasps; it is a generation instruction, not a guarantee that generated labels satisfy those requirements.
Table 24 reports increased unique-label counts across all datasets. Table 25 compares selected bimanual corpora and identifies YAM as largest among the listed open tabletop sets; its final row uses hours rather than trajectory count, so that row is not numerically comparable on the same axis.
Original selectable listing
You are an expert robot arm task instruction generator. You are provided with multiple demonstration videos, each showing the exact same task execution captured simultaneously from different camera angles. The task is to {original_instruction}. Based on these videos, generate a clear imperative instruction describing all of the actions performed by the robot arm. Carefully study the evolution of the state of the scene, to understand how the robot arm achieves the goal. Pay attention to the color and position of the objects the robot interacts with. Start directly with an action verb and make sure you do not miss any actions. Note that the robot arm is not always grasping an object and maybe pushing objects around instead. Verify any grasps by checking the gripper state, and only say the arm is grasping an object if this is shown in the wrist camera when the gripper is closed. Your prompts should be almost exactly {word_limit} words long.
Original selectable table
| Dataset | Original Instructions | Relabeled Instructions |
| MolmoAct2-BimanualYAM Dataset | 34 (0.1%) | 11448 (35%) |
| MolmoAct2-DROID Dataset | 49623 (67%) | 55905 (75%) |
| MolmoAct2-SO100/101 Dataset | 707 (1.5%) | 16205 (34%) |
| MolmoAct Dataset | 134 (1%) | 2959 (28%) |
| BC-Z | 104 (0.3%) | 9981 (25%) |
| Fractal | 598 (0.7%) | 16729 (19%) |
| Bridge | 19973 (37%) | 35096 (66%) |
| Total | 71121 (22%) | 146485 (46%) |
Original selectable table
| Dataset Name | Embodiment | Number of Trajectories | Open / Closed? |
| MolmoAct2-BimanualYAM Dataset (Ours) | Bimanual YAM | 34,500 | Open |
| AIST Bimanual Manipulation Dataset (Motoda et al., 2025) | ALOHA | 10,000 | Open |
| BiPlay Dataset (Dasari et al., 2025) | ALOHA | 7,000 | Open |
| RDT-1B Finetuning Dataset (Liu et al., 2024) | ALOHA | 6,000 | Open |
| ALOHA Unleashed Training Data (Zhao et al., 2024) | ALOHA | 26,000 | Closed |
| SARM RA-BC Dataset (Chen et al., 2025) | Bimanual YAM | 200 Hours | Closed |
Appendix E Limitations and Potential Solutions
The explicit limitations focus on action-chunk execution and deployment coverage. The benchmark gains should be read together with these constraints, rather than as evidence of a universally reliable robot controller.
Action chunks and the absence of real-time re-chunking.
Each prediction spans one second and executes open-loop before another observation is used. Independent chunk training has no continuity loss across boundaries and can produce velocity or acceleration jumps. The policy cannot react within a chunk to contact, disturbances or tracking error; the reported 55.79 actions/s is amortized generation throughput.
Embodiment-specific zero-shot deployment.
Out-of-the-box use is tied to YAM, SO-100/101 and DROID Franka, where substantial embodiment-specific data is available. Other robots require demonstrations and adaptation. Broader coverage is a future data-scaling direction rather than a demonstrated property of the released checkpoints.