Literature
Display
← All papers

GeomVLA: Unifying Scene, Motion, and Action in 3D

Ziyin Xiong · Nikolaos Gkanatsios · Moritz Reuss · Katerina Fragkiadaki

6 min · CondensedOriginal paper ↗
Contents

Overview

GeomVLA connects language-conditioned scene geometry, latent future motion, and action generation in one metric coordinate frame. The opening figure illustrates the shared representation.

Refer to caption
Figure 1: GeomVLA enables motion-guided 3D visual-language-action learning by bridging perception, latent future motion, and robot control in a shared 3D space. It lifts pretrained 2D VLM features into a geometrically grounded 3D representation, predicts latent 3D scene trajectories as an intermediate reasoning signal, and conditions a 3D action denoiser on the resulting scene-motion representation to generate robot actions. This design improves performance in both simulation and the real world. It also supports joint-angle action denoising by conditioning on scene-motion and end-effector pose-action features.

At a glance

GeomVLA lifts vision-language features into 3D, learns future point-motion representations, and conditions a geometric action denoiser on them. With benchmark demonstrations and no robot-action pretraining, it reaches 4.624 completed tasks on CALVIN, 98.4% on LIBERO, and 62.5% mean success across eight real tasks. Ablations support consistent geometry and learned motion features, with added inference cost.

1 Introduction

Image-plane reasoning can disconnect scene understanding from metric robot actions. GeomVLA represents the current scene, future point motion, and end-effector actions in the robot base frame so spatial relations remain consistent across perception and control.

Related references: [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29].

2 Related Work

Prior 3D policies ground actions geometrically, while motion-guided policies predict images, tracks, or latent futures. GeomVLA couples both through task-conditioned 3D motion features and geometry-aware action attention.

Related references: [10], [9], [30], [11], [31], [32], [33], [34], [14], [15], [18], [17], [35], [36], [37], [38], [39], [40], [41], [20], [42], [43], [44], [21], [26], [22], [24], [45], [46], [25], [47], [23], [48], [49], [50], [51], [52], [53], [54].

3 Method

Multi-view RGB-D observations, language, and proprioception enter a three-stage pathway: lifted 3D scene features, a point-trajectory denoiser, and an action denoiser. Intermediate motion features guide actions without requiring a fully generated future trajectory.

Refer to caption
Figure 2: GeomVLA architecture. Left: A shared 2D VLM encodes multi-view RGB observations and language instructions. Depth and camera geometry lift image features into 3D scene tokens with positions in the robot base frame. Middle: The 3D Scene Trajectory Denoiser is trained to predict future trajectories for front-view 3D anchors. During policy inference, one velocity-field evaluation provides intermediate motion features that are fused with the 3D scene tokens. Right: The 3D Action Denoiser predicts end-effector trajectory chunks conditioned on the fused scene-motion representation, language features, and robot proprioception. Optionally, the Joint Angle Denoiser uses the scene-motion representation and end-effector action features to predict joint-angle commands.

3.1 3D Scene Features

Florence-2 encodes image patches and language. Depth and camera calibration associate each visual token with a point in the robot base frame. Proprioceptive tokens are anchored at the current end-effector pose.

Related references: [6].

3.2 3D Scene Trajectory Denoiser

A 20×20 grid supplies 400 3D anchors. A flow-matching decoder predicts displacement increments conditioned on local geometry, image features, and language. SpatialTrackerV2 supplies pseudo-labels from demonstration videos. Motion-weighted training emphasizes moving points while retaining static structure.

Equations 1–2
ℱ={Δ​qi(t)}i=1,t=1N,H∈ℝN×H×3,\mathcal{F}=\{\Delta q_{i}^{(t)}\}_{i=1,t=1}^{N,H}\in\mathbb{R}^{N\times H\times 3}, (1)
ℒtraj=𝔼τ,ℱ0​[∑i=1N∑t=1Hwi,t​‖uθ​(ℱτ,τ,𝒞)i,t−ui,t⋆‖22],\mathcal{L}_{\mathrm{traj}}=\mathbb{E}_{\tau,\mathcal{F}_{0}}\left[\sum_{i=1}^{N}\sum_{t=1}^{H}w_{i,t}\left\|u_{\theta}(\mathcal{F}_{\tau},\tau,\mathcal{C})_{i,t}-u^{\star}_{i,t}\right\|_{2}^{2}\right], (2)

3.3 3D Action Denoiser

During policy inference, one trajectory-denoiser evaluation at the initial noise state supplies latent features; the model does not integrate a complete future trajectory. Temporal pooling gives one motion token per anchor, then geometry-aware attention fuses motion with scene tokens. A separate flow denoiser predicts end-effector translation and rotation, plus binary gripper state.

Equations 3–6
fa\displaystyle f_{\mathrm{a}} ←CrossAttn3​D​(fa,[fvmot;fℓ]),\displaystyle\leftarrow\mathrm{CrossAttn}_{3\mathrm{D}}\!\bigl(f_{\mathrm{a}},[f_{v}^{\mathrm{mot}};f_{\ell}]\bigr), (3)
fa\displaystyle f_{\mathrm{a}} ←SelfAttn3​D​(fa),\displaystyle\leftarrow\mathrm{SelfAttn}_{3\mathrm{D}}(f_{\mathrm{a}}), (4)
fa\displaystyle f_{\mathrm{a}} ←adaLN​-​FFN​(fa,λ,𝐩),\displaystyle\leftarrow\mathrm{adaLN\text{-}FFN}(f_{\mathrm{a}};\lambda,\mathbf{p}), (5)
ℒπ=𝔼λ,𝒜0​[‖vθ​(𝒜λ,λ,fa)−v⋆‖22+BCE⁡(fθopen,aopen)],\mathcal{L}_{\pi}=\mathbb{E}_{\lambda,\mathcal{A}_{0}}\left[\left\|v_{\theta}(\mathcal{A}_{\lambda},\lambda,f_{a})-v^{\star}\right\|_{2}^{2}+\mathrm{BCE}(f_{\theta}^{\mathrm{open}},a^{\mathrm{open}})\right], (6)

3D reasoning for joint-space control.

A downstream joint-angle denoiser can use the same scene-motion and intermediate end-effector features to produce executable joint commands, allowing the geometric representation to be tested under a matched output space.

3.4 Training

First, train the trajectory denoiser on point-track pseudo-labels with the vision-language backbone frozen. Second, jointly fine-tune the backbone, motion module, and action denoiser using only the action loss. Initialization includes pretrained Florence-2, but no robot-action pretraining.

Implementation details.

Videos are downsampled by three. Fifteen predicted displacement steps span 45 original frames; coordinates remain in the robot base frame.

Related references: [55].

4 Experiments

The evaluation tests motion supervision, shared geometry, simulation performance, and transfer to tabletop manipulation.

4.1 Simulation Experiments

Simulation covers CALVIN, LIBERO, and RoboTwin2.0. Full GeomVLA and the main π0.5 baseline train on all fifty RoboTwin tasks; several variants train only on a five-task subset. Comparisons across those training sets are not fully controlled ablations.

Related references: [56], [27], [28], [29].

Table 1: Comparison with prior methods on simulation benchmarks. GeomVLA-JA denotes the joint-angle action variant conditioned on 3D scene-motion features and intermediate end-effector action features; 2D-JA is a 2D VLA baseline using the same VLM, joint-angle action denoiser, and training data as GeomVLA-JA; GeomVLA w/o motion removes the 3D Scene Trajectory Denoiser while retaining the 3D action policy. Pretrained indicates large-scale robot-action pretraining. For RoboTwin2.0, (a)–(e) are a five-task evaluation subset selected following SimpleVLA-RL [56], each reported as the mean success rate across the Easy and Hard settings: (a) beat_block_hammer, (b) pick_dual_bottles, (c) move_can_pot, (d) handover_mic, and (e) stack_bowls_two. Tasks (a)–(b), (c)–(d), and (e) are short-, medium-, and long-horizon tasks, with approximately 100, 200, and 300 steps, respectively. We fine-tune π0.5\pi_{0.5} and train full GeomVLA on all 50 tasks; GeomVLA w/o motion, 2D-JA, and GeomVLA-JA are trained only on the five selected tasks. The five-task comparison therefore uses different training sets and is not a fully controlled ablation.
Method Pretrained Params CALVIN LIBERO RoboTwin2.0
ABC_D Spatial Object Goal Long Avg. a b c d e Avg.
3D VLA
3D-VLA Yes – 0.707 – – – – – – – – – – –
SpatialVLA Yes 4B – 88.2 89.9 78.6 55.5 78.1 – – – – – –
4D-VLA Yes 4B+ – 88.9 95.2 90.0 79.1 88.6 – – – – – –
PoseVLA Yes 3B+ – 96.5 98.0 97.1 92.4 96.0 93.5 87.0 58.0 93.0 96.5 85.6
Video/Motion-Guided
TraceVLA Yes 7B – 84.6 85.2 75.1 54.1 74.8 – – – – – –
mimic-video Yes 2B+ – 94.2 96.8 90.6 – 93.9 – – – – – –
WorldVLA No 7B – 87.6 96.2 83.4 60.0 81.8 – – – – – –
UniVLA Yes 7B 3.801 96.5 96.8 95.6 92.0 95.2 – – – – – –
CoT-VLA Yes 7B – 87.5 91.6 87.6 69.0 81.1 – – – – – –
FlowVLA Yes 8.5B – 93.2 95.0 91.6 72.6 88.1 – – – – – –
LaMP Yes 5B – 99.4 99.8 97.4 96.7 98.3 – – – – – –
RoboFlow4D Yes 0.76B+ – 90.2 97.0 88.4 75.2 87.7 – – – – – –
Others
π0.5\pi_{0.5} Yes 3B+ 3.964 98.8 98.2 98.0 92.4 96.8 82.0 94.0 90.0 47.0 93.0 81.2
OpenVLA-OFT Yes 7B 4.279 97.6 98.4 97.9 94.5 97.1 28.1 29.7 28.1 45.3 40.6 34.4
FLOWER Yes 0.95B 4.480 97.2 99.3 96.9 94.5 97.0 – – – – – –
SimpleVLA-RL Yes 7B – 99.4 99.1 99.2 98.5 99.1 87.5 68.3 61.2 89.2 75.8 76.4
X-VLA Yes 0.9B 4.430 98.2 98.6 97.8 97.6 98.1 48.0 28.5 47.0 69.0 46.5 47.8
2D-JA No 1B 4.195 – – – – – 39.0 77.0 88.0 74.0 53.0 66.2
GeomVLA-JA No 1.3B 4.508 – – – – – 71.0 75.0 91.0 95.0 76.0 81.6
GeomVLA w/o motion No 1.2B 4.508 94.8 99.2 88.2 91.8 93.5 45.0 96.0 83.0 83.0 43.0 70.0
GeomVLA No 1.2B 4.624 98.6 99.8 97.6 97.6 98.4 89.0 98.0 99.0 55.0 83.0 84.8

Results.

GeomVLA scores 4.624 on CALVIN and 98.4% on LIBERO. Across all fifty RoboTwin tasks, Easy/Hard success is 78.6%/76.2% versus 75.9%/75.7% for fully fine-tuned π0.5. The five-task results in the main table should be distinguished from this full-benchmark comparison.

Related references: [3], [57], [7], [56], [8], [14], [15], [16], [58], [22], [59], [20], [41], [60], [21], [46], [25].

Refer to caption
Figure 3: Simulation examples from CALVIN, LIBERO, and RoboTwin2.0. Left: predicted 3D point trajectories from the Scene Trajectory Denoiser. Right: corresponding closed-loop policy rollouts.

4.2 Ablation Studies

On CALVIN, adding motion raises the 3D policy from 4.508 to 4.624. Fully denoised trajectories, late latents, or encoder features perform worse than early motion latents. Mixing 2D motion with 3D actions drops performance to 4.085. Applying one shared rigid transform preserves most performance (4.596), suggesting consistency matters more than the particular coordinate frame.

Table 2: CALVIN ABC-D ablations. The metric is the mean number of consecutive tasks completed in each five-instruction sequence. The full model performs best among the evaluated variants.
Variant Description Perf.
GeomVLA w/o motion Removes the 3D Scene Trajectory Denoiser. 4.508
2DGeomVLA w/o motion Uses a 2D image-based policy without future-motion reasoning. 4.411
2DGeomVLA Uses 2D future-motion reasoning with a 2D action denoiser. 4.437
2DGeomVLA LaMP-style Uses pixel (u,v)(u,v) coordinates and metric depth for scene motion. 4.462
GeomVLA w/ 2D motion Uses image-space future motion with the 3D action denoiser. 4.085
GeomVLA w/ denoised trace Conditions action denoising on the fully denoised 3D trajectory. 4.047
GeomVLA w/ denoised latent Conditions action denoising on the fully denoised motion latent. 4.334
GeomVLA w/ partial denoising Reads the motion latent at τ=0.1\tau=0.1 instead of pure noise. 4.484
GeomVLA encoder-only Uses only the scene trajectory encoder, without velocity prediction. 4.443
GeomVLA w/ arbitrary frame Expresses scene, motion, and action in one arbitrary rigid frame. 4.596
GeomVLA Unifies scene, future motion, and action in a shared 3D frame. 4.624

4.3 Real-World Experiments

Real-world evaluation uses a single follower arm on an ALOHA2 platform.

Setup.

Teleoperation provides demonstrations. Two elevated depth cameras and a wrist camera observe the tabletop workspace.

Tasks.

Eight tasks cover insertion, stacking, bottle placement, reorientation, transfer, uncapping, color order, and triangular arrangement. Three use fifty demonstrations each and five use twenty. Each variant trains one multi-task policy on the same demonstrations; the main π0.5 comparison additionally differs in pretraining and action parameterization.

Results.

With twenty evaluations per task, GeomVLA averages 62.5% success, beats its no-motion version on every task, and exceeds π0.5 on seven of eight. Color ordering remains difficult. Illustrated recoveries are qualitative examples, not separately quantified robustness tests.

Refer to caption
Figure 4: Real-world evaluation setup and tasks. We evaluate GeomVLA on an ALOHA2 platform on single-arm tabletop tasks. The left panel shows representative rollouts on three real-world tasks: placing two bottles into two cups, inserting a marker into a cup, and stacking two blocks.
Table 3: Real-world success rates on ALOHA2. Each entry reports the number of successful trials among 20 evaluations. GeomVLA-JA denotes the joint-angle action variant conditioned on 3D scene-motion features and intermediate end-effector action features; 2D-JA uses the same VLM, joint-angle denoiser, and training data but performs perception and motion reasoning in 2D; GeomVLA w/o motion removes the 3D Scene Trajectory Denoiser; GeomVLA is the full Cartesian-action model. † denotes the three tasks with 50 demonstration rollouts each; the remaining five tasks use 20 demonstration rollouts.
Task π0.5\pi_{0.5} 2D-JA GeomVLA-JA GeomVLA w/o motion GeomVLA
Insert marker† 9/20 3/20 13/20 3/20 5/20
Stack blocks† 2/20 9/20 17/20 15/20 19/20
Place bottles† 10/20 16/20 16/20 14/20 17/20
Stand bottle 7/20 6/20 13/20 13/20 17/20
Transfer marker 2/20 0/20 13/20 10/20 12/20
Uncap marker 15/20 12/20 18/20 15/20 20/20
Rank by color 0/20 0/20 1/20 0/20 2/20
Place in triangle 2/20 0/20 4/20 1/20 8/20
Average 29.4% 28.8% 59.4% 44.4% 62.5%

4.4 3D Reasoning for Joint-Angle Robot Control

GeomVLA-JA and 2D-JA share the backbone, joint-angle output head, and training data, helping isolate the representation benefit from the choice of output commands.

Results.

GeomVLA-JA exceeds 2D-JA on CALVIN (4.508 versus 4.195), the five-task RoboTwin subset (81.6% versus 66.2%), and real tasks (59.4% versus 28.8%). It performs particularly well on marker insertion: 13/20 successes versus 5/20 for the Cartesian model.

5 Limitations

The method depends on accurate depth and calibration, uses benchmark-scale demonstrations, and has not established broad cross-embodiment generalization. Its latent motion representation is not explicit verbal or hierarchical reasoning. Adding the motion pathway increases inference time.

6 Conclusion

A shared 3D representation improves the tested policies across Cartesian and joint-angle outputs. Learned early motion features appear more useful for action generation than fully reconstructed trajectories.

Appendix

The appendix retains configurations, complete RoboTwin results, stability checks, motion analyses, and detailed task descriptions.

Model configurations and compute.

Second-stage training takes about fifteen hours on four L40S GPUs for CALVIN, 25 hours on eight A100s for LIBERO, and 48 hours on eight A100s for the five-task RoboTwin setting. Table 4 records benchmark-specific architecture and optimization choices.

Lifting 2D VLM features into 3D.

Bilinearly interpolated depth at each patch center, camera intrinsics, and extrinsics lift image tokens into metric 3D positions.

3D scene trajectory encoder.

Local attention over a fixed nearest-neighbor graph combines centered coordinates and semantic descriptors. Relative-position biases preserve local geometry; global visual and language tokens condition the decoder.

Equations 7–9
𝐮¯i=[ui,vi,1]⊤,𝐪i=Rc→b​(D⁡(𝐮i)​K−1​𝐮¯i)+𝐭c→b,𝐪~i=𝐪i−1N​∑j=1N𝐪j.\bar{\mathbf{u}}_{i}=[u_{i},v_{i},1]^{\top},\qquad\mathbf{q}_{i}=R_{c\rightarrow b}\bigl(D(\mathbf{u}_{i})K^{-1}\bar{\mathbf{u}}_{i}\bigr)+\mathbf{t}_{c\rightarrow b},\qquad\tilde{\mathbf{q}}_{i}=\mathbf{q}_{i}-\frac{1}{N}\sum_{j=1}^{N}\mathbf{q}_{j}. (7)
hi(0)=MLP⁡([𝐪~i;𝐮i;xi])∈ℝD.h_{i}^{(0)}\;=\;\mathrm{MLP}\!\big([\,\tilde{\mathbf{q}}_{i};\;\mathbf{u}_{i};\;x_{i}\,]\big)\;\in\;\mathbb{R}^{D}. (8)
hi(ℓ+1)=hi(ℓ)+∑j∈𝒩k​(i)αi​j(ℓ)​Wv(ℓ)​hj(ℓ),αi​j(ℓ)∝exp⁡((Wq(ℓ)​hi(ℓ))⊤​(Wk(ℓ)​hj(ℓ))dh+b⁡(𝐪~j−𝐪~i)).h_{i}^{(\ell+1)}\;=\;h_{i}^{(\ell)}+\!\!\sum_{j\in\mathcal{N}_{k}(i)}\!\!\alpha_{ij}^{(\ell)}\,W_{v}^{(\ell)}h_{j}^{(\ell)},\qquad\alpha_{ij}^{(\ell)}\propto\exp\!\Bigg(\frac{\bigl(W_{q}^{(\ell)}h_{i}^{(\ell)}\bigr)^{\!\top}\!\bigl(W_{k}^{(\ell)}h_{j}^{(\ell)}\bigr)}{\sqrt{d_{h}}}+b(\tilde{\mathbf{q}}_{j}-\tilde{\mathbf{q}}_{i})\Bigg). (9)

3D scene trajectory decoder.

The trajectory decoder alternates spatial attention over Morton-ordered windows, temporal attention, and cross-attention to context. Shifted windows and axis permutations spread spatial information. Invalid or low-confidence tracks are masked, and a motion-sensitive weighting retains nonzero supervision for static points. The equations and configuration table preserve the implementation details.

Equations 10–14
Fτ=(1−τ)​F0+τ​F,u⋆=F−F0.F_{\tau}\;=\;(1-\tau)\,F_{0}+\tau\,F,\qquad u^{\star}\;=\;F-F_{0}. (10)
Zi,t(0)=WF​Fτ,i,t+ci+et+ϕ⁡(τ).Z_{i,t}^{(0)}\;=\;W_{F}\,F_{\tau,i,t}\;+\;c_{i}\;+\;e_{t}\;+\;\phi(\tau). (11)
SerSpatial​(Z)π⁡(i),t=Attn⁡(Zπ⁡(i),t,{Zπ⁡(j),t:j∈𝒲P​(i)}),𝒲P​(i)={j:⌊j/P⌋=⌊i/P⌋}.\mathrm{SerSpatial}(Z)_{\pi(i),t}\;=\;\mathrm{Attn}\!\big(Z_{\pi(i),t}\,;\;\{Z_{\pi(j),t}:j\in\mathcal{W}_{P}(i)\}\big),\qquad\mathcal{W}_{P}(i)=\big\{j:\lfloor j/P\rfloor=\lfloor i/P\rfloor\big\}. (12)
Z→SerSpatialZ→TemporalZ→Cross⁡([Vg;E])Z→FFNZ.Z\;\xrightarrow{\;\mathrm{SerSpatial}\;}\;Z\;\xrightarrow{\;\mathrm{Temporal}\;}\;Z\;\xrightarrow{\;\mathrm{Cross}([V_{g};E])\;}\;Z\;\xrightarrow{\;\mathrm{FFN}\;}\;Z. (13)
ℒ=𝔼τ,F0[∑i,twi,t∥uθ(Fτ,τ,𝒞)i,t−ui,t⋆∥22],\mathcal{L}\;=\;\mathbb{E}_{\tau,F_{0}}\!\left[\,\sum_{i,t}w_{i,t}\,\bigl\lVert u_{\theta}(F_{\tau},\tau,\mathcal{C})_{i,t}-u^{\star}_{i,t}\bigr\rVert_{2}^{2}\right], (14)
Table 4: Model configurations across simulation benchmarks and real-world experiments. The 3D action denoiser shared attention layers are the action–scene fusion blocks that are shared by translation and rotation prediction. Rotation format is the policy rotation representation, with RoboTwin2.0 rotations converted to 6D internally; action horizon is the number of future actions predicted per policy query, and embodiment specifies the controlled arm setup.
Module CALVIN LIBERO RoboTwin2.0 – five-task ablations Real-world
Trajectory Denoiser Hidden dim 512 384 320 128
Encoder layers 6 6 6 4
Decoder layers 8 8 8 8
Attention heads 16 8 8 8
Batch size 128 128 128 128
Optimizer AdamW AdamW AdamW AdamW
Learning rate 1e-4 1e-4 1e-4 1e-4
Iterations 20k 20k 20k 20k
Action Denoiser Hidden dim 960 912 912 256
Shared attention layers 16 16 16 16
Attention heads 16 16 16 16
Batch size 32 128 64 32
Optimizer AdamW AdamW AdamW AdamW
Learning rate 2e-5 5e-5 4e-5 2e-5
Iterations 50k 80k 300k 50k
Rotation format Euler Axis-angle 6D rotation 6D rotation
Action horizon 10 10 16 8
Embodiment Single-arm Single-arm Bimanual Single-arm

RoboTwin2.0 training protocol and full results.

Full RoboTwin training uses 27,500 demonstrations: fifty clean and five hundred randomized examples per task. Table 5 reports all fifty tasks, each evaluated in fifty Easy and fifty Hard rollouts.

Table 5: Per-task success rates on 50 RoboTwin2.0 tasks. Each task is evaluated under both Easy and Hard domain randomization, with 50 rollouts per setting. We report success rates as percentages. Average: π0.5\boldsymbol{\pi_{0.5}} Easy 75.9 / Hard 75.7, GeomVLA Easy 78.6 / Hard 76.2.
Task 𝝅0.5\boldsymbol{\pi_{0.5}} GeomVLA Task 𝝅0.5\boldsymbol{\pi_{0.5}} GeomVLA
Easy Hard Easy Hard Easy Hard Easy Hard
Adjust Bottle 86 92 100 100 Place Can Basket 70 70 84 76
Beat Block Hammer 84 80 88 90 Place Cans Plasticbox 74 86 94 92
Blocks Ranking RGB 88 84 46 44 Place Container Plate 96 98 100 98
Blocks Ranking Size 52 38 24 32 Place Dual Shoes 62 64 84 86
Click Alarmclock 94 98 98 96 Place Empty Cup 90 90 100 96
Click Bell 98 98 100 100 Place Fan 88 84 88 86
Dump Bin Bigbin 84 90 72 68 Place Mouse Pad 44 36 88 90
Grab Roller 98 96 100 100 Place Object Basket 70 80 48 56
Handover Block 42 32 14 10 Place Object Scale 80 70 96 98
Handover Mic 42 52 72 38 Place Object Stand 86 94 96 98
Hanging Mug 16 14 6 4 Place Phone Stand 64 66 76 84
Lift Pot 88 88 32 18 Place Shoe 88 84 96 100
Move Can Pot 92 88 100 98 Press Stapler 70 78 70 78
Move Pillbottle Pad 80 88 96 94 Put Bottles Dustbin 34 38 34 18
Move Playingcard Away 74 76 98 96 Put Object Cabinet 56 60 62 48
Move Stapler Pad 56 64 72 76 Rotate QRcode 96 84 80 80
Open Laptop 90 92 94 92 Scan Object 62 56 62 60
Open Microwave 82 80 94 82 Shake Horizontally 96 98 100 100
Pick Diverse Bottles 86 84 86 84 Shake Bottle 96 98 100 100
Pick Dual Bottles 94 94 100 96 Stack Blocks Three 56 60 46 32
Place A2B Left 84 90 90 90 Stack Blocks Two 92 92 74 88
Place A2B Right 82 72 96 90 Stack Bowls Three 78 64 46 36
Place Bread Basket 74 68 86 86 Stack Bowls Two 88 98 84 82
Place Bread Skillet 72 66 92 80 Stamp Seal 88 74 96 96
Place Burger Fries 78 82 96 94 Turn Switch 56 58 72 72

Two-seed stability check on CALVIN and LIBERO.

A second seed gives CALVIN 4.609 versus 4.624 and LIBERO 98.0% versus 98.4%. This limited stability check does not quantify uncertainty for every comparison.

Evaluation of the 3D scene trajectory prediction horizon.

Increasing the trajectory horizon from fifteen to 32 raises mean error from 2.46 to 3.35 cm and endpoint error from 4.61 to 6.43 cm over the common evaluation window. CALVIN drops from 4.624 to 4.272.

Table 6: Effect of trajectory-prediction horizon on CALVIN. Errors are computed on held-out valid moving points against SpatialTrackerV2 pseudo-reference tracks over the first 15 future steps for both models. In this two-setting comparison, the longer training horizon yields higher trajectory error and lower downstream performance.
Horizon Mean error (cm) ↓\downarrow Endpoint error (cm) ↓\downarrow Perf. ↑\uparrow
H=15H=15 2.46 4.61 4.624
H=32H=32 3.35 6.43 4.272

Instruction conditioning for the 3D Scene Trajectory Denoiser.

Replacing only the task instruction increases endpoint error on all three benchmarks, supporting language dependence of the learned motion predictions.

Table 7: Instruction sensitivity of predicted 3D scene trajectories. Endpoint error is measured in centimeters on held-out valid moving points against SpatialTrackerV2 pseudo-reference tracks at H=15H=15. Replacing the matched instruction with a mismatched instruction increases the error on all three benchmarks.
Benchmark Matched ↓\downarrow Mismatched ↓\downarrow Increase
CALVIN 4.61 8.01 +3.40+3.40
LIBERO 9.12 14.56 +5.44+5.44
RoboTwin2.0 5.77 7.41 +1.64+1.64

Control for model capacity.

Replacing learned motion features with a frozen random module of equal size reduces CALVIN from 4.624 to 4.443. Added parameter count alone does not explain the measured gain.

Inference latency.

On one L40S, a ten-action chunk takes 280.0±14.0 ms with motion versus 219.0±11.3 ms without it: 27.9% overhead. The action denoiser dominates total time. Measurements cover four evaluation sequences per model after warmup.

Ablation variant descriptions.

Table 8 defines the exact changes behind each ablation, including action geometry, motion-token extraction, and coordinate alignment.

Table 8: Ablation model descriptions. Detailed descriptions of the model variants evaluated in the ablation study. Each variant modifies one aspect of GeomVLA, including future-motion reasoning, action denoising, motion-token extraction, or coordinate-frame alignment, while the corresponding quantitative results are reported in the main paper.
Variant Description
GeomVLA w/o motion Removes the 3D Scene Trajectory Denoiser and directly predicts actions with the 3D action denoiser.
2DGeomVLA w/o motion Uses a 2D image-based action denoiser without future-motion reasoning.
2DGeomVLA Uses 2D future-motion reasoning with a 2D action denoiser.
2DGeomVLA LaMP-style Uses image-plane (u,v)(u,v) coordinates and metric depth for motion reasoning instead of metric 3D points.
GeomVLA w/ 2D motion Uses image-space future motion with the 3D action denoiser.
GeomVLA w/ denoised trace Conditions action denoising on the fully denoised 3D trajectory.
GeomVLA w/ denoised latent Conditions action denoising on the fully denoised motion latent.
GeomVLA w/ partial denoising Reads the motion latent at τ=0.1\tau=0.1 instead of pure noise.
GeomVLA encoder-only Uses only the trajectory-denoiser encoder, without velocity prediction.
GeomVLA w/ arbitrary frame Expresses scene, motion, and action in a shared arbitrary rigid frame.
GeomVLA Unifies scene, future motion, and action in a shared 3D frame.

Real-world task descriptions.

The eight tasks test precise orientation, small-object manipulation, color grounding, and sequential placement. Marker insertion, block stacking, and paired bottle placement use fifty demonstrations; the other tasks use twenty. Figure 5 shows successful rollouts with varied objects, distractors, and layouts.

Refer to caption
Figure 5: Qualitative GeomVLA rollouts on five additional real-world tasks. These tasks complement those shown in Figure 4. Each row shows a temporal sequence from one successful rollout. From top to bottom, the tasks are Rank by color, Transfer marker, Place in triangle, Uncap marker, and Stand bottle upright.