Literature
Display
← All papers

Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Cheng Chi · Zhenjia Xu · Siyuan Feng · Eric Cousineau · Yilun Du · Benjamin Burchfiel · Russ Tedrake · Shuran Song

· The International Journal of Robotics Research ↗

Robotics12 min text · CondensedOriginal paper ↗
Contents

Overview

Figure 1 contrasts direct action prediction, energy-based optimization, and iterative action denoising. The edition follows the extended arXiv revision v5, including the bimanual experiments and appendices.

[Uncaptioned image]
Figure 1: Policy Representations. a) Explicit policy with different types of action representations. b) Implicit policy learns an energy function conditioned on both action and observation and optimizes for actions that minimize the energy landscape c) Diffusion policy refines noise into actions via a learned gradient field. This formulation provides stable training, allows the learned policy to accurately model multimodal action distributions, and accommodates high-dimensional action sequences.

At a glance

Diffusion Policy learns to denoise future robot actions conditioned on recent observations. Predicting an action sequence and repeatedly executing only a prefix combines temporally consistent behavior with feedback. Across the reported manipulation benchmarks, the best diffusion variants improve over the best baseline by an average of 46.9% relative, not 46.9 percentage points. Real robot studies cover pushing, mug reorientation, sauce manipulation, and three bimanual skills. The advantages come with iterative-inference cost and the data limitations of behavior cloning.

1 Introduction

Robot demonstrations contain multiple valid actions, correlated motion, and precise contact requirements. Instead of selecting a fixed number of action modes, Diffusion Policy learns a conditional denoising field over an entire action sequence. Its main design choices are receding-horizon execution, conditioning on encoded images without generating future observations, and either a temporal convolutional network or a diffusion transformer. This extended version adds control-theoretic analysis, visual-encoder ablations, and bimanual experiments to the conference paper.

Refer to caption
Figure 2: Diffusion Policy Overview a) General formulation. At time step tt, the policy takes the latest ToT_{o} steps of observation data OtO_{t} as input and outputs TaT_{a} steps of actions AtA_{t}. b) In the CNN-based Diffusion Policy, FiLM (Feature-wise Linear Modulation) [39] conditioning of the observation feature OtO_{t} is applied to every convolution layer, channel-wise. Starting from 𝐀tK\mathbf{A}^{K}_{t} drawn from Gaussian noise, the output of noise-prediction network ϵθ\epsilon_{\theta} is subtracted, repeating KK times to get 𝐀t0\mathbf{A}^{0}_{t}, the denoised action sequence. c) In the Transformer-based [60] Diffusion Policy, the embedding of observation 𝐎t\mathbf{O}_{t} is passed into a multi-head cross-attention layer of each transformer decoder block. Each action embedding is constrained to only attend to itself and previous action embeddings (causal attention) using the attention mask illustrated.

2 Diffusion Policy Formulation

The policy is a conditional denoising diffusion probabilistic model: training learns to remove noise from demonstrated actions, and inference starts from noise to sample an observation-conditioned action sequence.

2.1 Denoising Diffusion Probabilistic Models

Start with Gaussian noise and repeatedly apply a learned noise predictor. The update combines denoising, scaling, and optional stochastic noise according to the diffusion schedule. The paper interprets the update as a noisy gradient step on an energy landscape; the preserved equations specify that relationship and its coefficients.

Equations 1–2
xk−1=α(xk−γϵθ(𝐱k,k)+𝒩(0,σ2I)),\textbf{x}^{k-1}=\alpha(\textbf{x}^{k}-\gamma\epsilon_{\theta}(\mathbf{x}^{k},k)+\mathcal{N}\bigl(0,\sigma^{2}I\bigl)),\vskip-2.84526pt (1)
𝐱′=𝐱−γ∇E(𝐱),\mathbf{x}^{\prime}=\mathbf{x}-\gamma\nabla E(\mathbf{x}),\vskip-2.84526pt (2)

2.2 DDPM Training

Training samples a demonstration, diffusion step, and noise realization, then minimizes mean squared error between the sampled noise and the network prediction. This avoids explicitly normalizing an energy model. The denoising objective is related to a variational bound on the divergence between data and model distributions. [21]

Equation 3
ℒ=M​S​E​(ϵk,ϵθ​(𝐱0+ϵk,k))\mathcal{L}=MSE(\mathbf{\epsilon}^{k},\epsilon_{\theta}(\mathbf{x}^{0}+\mathbf{\epsilon}^{k},k)) (3)

2.3 Diffusion for Visuomotor Policy Learning

At each planning step, the policy observes the latest To frames, predicts Tp future actions, executes Ta of them, then replans. This separates prediction horizon from open-loop execution duration. The diffusion model learns actions conditioned on observations, rather than a joint distribution over actions and future states. Images can therefore be encoded once per inference call and reused through all denoising iterations. Warm-starting from the previous prediction is also compatible with receding-horizon control.

Equations 4–5
𝐀tk−1=α(𝐀tk−γϵθ(𝐎t,𝐀tk,k)+𝒩(0,σ2I))\mathbf{A}^{k-1}_{t}=\alpha(\mathbf{A}^{k}_{t}-\gamma\epsilon_{\theta}(\mathbf{O}_{t},\mathbf{A}^{k}_{t},k)+\mathcal{N}\bigl(0,\sigma^{2}I\bigl)) (4)
ℒ=M​S​E​(ϵk,ϵθ​(𝐎t,𝐀t0+ϵk,k))\mathcal{L}=MSE(\mathbf{\epsilon}^{k},\epsilon_{\theta}(\mathbf{O}_{t},\mathbf{A}^{0}_{t}+\mathbf{\epsilon}^{k},k)) (5)

3 Key Design Decisions

The important choices are the action denoiser, visual encoder, noise schedule, and inference sampler. Their suitability depends on action frequency, data quantity, and control latency.

3.1 Network Architecture Options

The convolutional denoiser uses temporal convolutions and FiLM observation conditioning. It is relatively robust to hyperparameters but can oversmooth rapidly changing actions. The transformer predicts noise for action tokens using causal action attention and cross-attention to observations; it often performs better on difficult state-based tasks but needs more tuning. The authors recommend beginning with the convolutional model and trying the transformer when temporal complexity or fast changes limit performance.

3.2 Visual Encoder

Each camera view has a separate image encoder; embeddings from the observation window are concatenated. The default ResNet-18 is trained jointly with the policy. Spatial softmax pooling preserves localization information, and GroupNorm replaces BatchNorm to work reliably with exponential moving-average training.

3.3 Noise Schedule

The noise schedule controls how different temporal frequencies of the action signal are learned. The tested tasks favor the squared-cosine schedule from improved denoising diffusion probabilistic models.

3.4 Accelerating Inference for Real-time Control

DDIM allows fewer denoising steps at inference than during training. This section reports 100 training steps and 10 inference steps, giving approximately 0.1 seconds of inference latency on a GeForce RTX 3080. Appendix A.4 instead specifies 16 real-world inference steps; both statements are retained as a source discrepancy. These are model-inference measurements, not a guarantee for arbitrary high-rate control. [52]

4 Intriguing Properties of Diffusion Policy

The following analyses explain observed multimodality, position-control performance, temporal consistency, and stable training. They combine experiments with mechanistic interpretations rather than proving universal advantages.

4.1 Model Multi-Modal Action Distributions

Random initialization and stochastic denoising can reach different valid action modes. In the illustrated Push-T state, the policy approaches from either side while committing to one route within a rollout. The comparison is a selected-state demonstration of multimodality, not a complete measure of distribution coverage.

Original figure
Figure 3: Multimodal behavior. At the given state, the end-effector (blue) can either go left or right to push the block. Diffusion Policy learns both modes and commits to only one mode within each rollout. In contrast, both LSTM-GMM Mandlekar et al. (2021) and IBC Florence et al. (2021) are biased toward one mode, while BET Shafiullah et al. (2022) fails to commit to a single mode due to its lack of temporal action consistency. Actions generated by rolling out 40 steps for the best-performing checkpoint.

4.2 Synergy with Position Control

Position commands improve Diffusion Policy in the tested ablations, whereas the evaluated recurrent and transformer baselines favor velocity commands. The authors hypothesize that diffusion handles positional multimodality better, while position control accumulates less integration error across a predicted sequence. These mechanisms are explanations proposed for the measured result.

Original figure
Figure 4: Velocity v.s. Position Control. The performance difference when switching from velocity to position control. While both BCRNN and BET performance decrease, Diffusion Policy is able to leverage the advantage of position and improve its performance.

4.3 Benefits of Action-Sequence Prediction

Predicting a sequence jointly discourages switching between incompatible modes on consecutive steps. It also represents pauses in context: a stationary interval can be followed by purposeful motion instead of becoming an absorbing behavior. Executing too long a prefix reduces responsiveness, so sequence prediction still requires frequent replanning.

Original figure
Figure 5: Diffusion Policy Ablation Study. Change (difference) in success rate relative to the maximum for each task is shown on the Y-axis. Left: trade-off between temporal consistency and responsiveness when selecting the action horizon. Right: Diffusion Policy with position control is robust against latency. Latency is defined as the number of steps between the last frame of observations to the first action that can be executed.

4.4 Training Stability

Implicit behavior cloning estimates an energy-model normalization term using negative samples, a source of instability in the tested baseline. Diffusion instead learns the score, which is independent of that normalization constant. The preserved derivation explains why neither its training nor action inference needs to evaluate the normalizer. The observed stability also makes checkpoint selection easier on hardware.

Equations 6–8
pθ​(𝐚|𝐨)=e−Eθ​(𝐨,𝐚)Z⁡(𝐨,θ)p_{\theta}(\mathbf{a}|\mathbf{o})=\frac{e^{-E_{\theta}(\mathbf{o},\mathbf{a})}}{Z(\mathbf{o},\theta)}\vskip-2.84526pt (6)
ℒi​n​f​o​N​C​E=−log⁡(e−Eθ​(𝐨,𝐚)e−Eθ​(𝐨,𝐚)+∑j=1Nn​e​ge−Eθ​(𝐨,𝐚~j))\mathcal{L}_{infoNCE}=-\log(\frac{e^{-E_{\theta}(\mathbf{o},\mathbf{a})}}{e^{-E_{\theta}(\mathbf{o},\mathbf{a})}+{\color[rgb]{1,0,0}\sum^{N_{neg}}_{j=1}}e^{-E_{\theta}(\mathbf{o},{\color[rgb]{1,0,0}\widetilde{\mathbf{a}}^{j}})}}) (7)
∇𝐚​log​p​(𝐚|𝐨)=−∇𝐚Eθ​(𝐚,𝐨)−∇𝐚​log​Z​(𝐨,θ)⏟=0≈−ϵθ​(𝐚,𝐨)\nabla_{\mathbf{a}}\log p(\mathbf{a}|\mathbf{o})=-\nabla_{\mathbf{a}}E_{\theta}(\mathbf{a},\mathbf{o})-\underbrace{\nabla_{\mathbf{a}}\log Z(\mathbf{o},\theta)}_{=0}\approx-\epsilon_{\theta}(\mathbf{a},\mathbf{o}) (8)
Original figure
Figure 6: Training Stability. Left: IBC fails to infer training actions with increasing accuracy despite smoothly decreasing training loss for energy function. Right: IBC’s evaluation success rate oscillates, making checkpoint selection difficult (evaluated using policy rollouts in simulation).

4.5 Connections to Control Theory

For a linear system demonstrated by a linear feedback controller, the ideal one-step denoiser recovers that controller, and DDIM converges to its action. Predicting multiple future actions additionally requires the task-relevant closed-loop dynamics. This is a limiting-case sanity check under linearity and optimal-denoiser assumptions; nonlinear plants or policies can again require multimodal predictions.

Equations 9
𝐬t+1=𝐀𝐬t+𝐁𝐚t+𝐰t,𝐰t∼𝒩⁡(0,Σw).\displaystyle{\bf s}_{t+1}={\bf A}{\bf s}_{t}+{\bf B}{\bf a}_{t}+{\bf w}_{t},\qquad{\bf w}_{t}\sim\mathcal{N}(0,\Sigma_{w}).
ℒ=M​S​E​(ϵk,ϵθ​(𝐬t,−𝐊𝐬t+ϵk,k))\mathcal{L}=MSE(\mathbf{\epsilon}^{k},\epsilon_{\theta}(\mathbf{s}_{t},-{\bf K}{\bf s}_{t}+\mathbf{\epsilon}^{k},k)) (9)
ϵθ​(𝐬,𝐚,k)=1σk​[𝐚+𝐊𝐬],\epsilon_{\theta}({\bf s},{\bf a},k)=\frac{1}{\sigma_{k}}[{\bf a}+{\bf K}{\bf s}],
Original figure
Lift Can Square Transport ToolHang Push-T
ph mh ph mh ph mh ph mh ph ph
LSTM-GMM 1.00/0.96 1.00/0.93 1.00/0.91 1.00/0.81 0.95/0.73 0.86/0.59 0.76/0.47 0.62/0.20 0.67/0.31 0.67/0.61
IBC 0.79/0.41 0.15/0.02 0.00/0.00 0.01/0.01 0.00/0.00 0.00/0.00 0.00/0.00 0.00/0.00 0.00/0.00 0.90/0.84
BET 1.00/0.96 1.00/0.99 1.00/0.89 1.00/0.90 0.76/0.52 0.68/0.43 0.38/0.14 0.21/0.06 0.58/0.20 0.79/0.70
DiffusionPolicy-C 1.00/0.98 1.00/0.97 1.00/0.96 1.00/0.96 1.00/0.93 0.97/0.82 0.94/0.82 0.68/0.46 0.50/0.30 0.95/0.91
DiffusionPolicy-T 1.00/1.00 1.00/1.00 1.00/1.00 1.00/0.94 1.00/0.89 0.95/0.81 1.00/0.84 0.62/0.35 1.00/0.87 0.95/0.79
Table 1: Behavior Cloning Benchmark (State Policy) We present success rates with different checkpoint selection methods in the format of (max performance) / (average of last 10 checkpoints), with each averaged across 3 training seeds and 50 different environment initial conditions (150 in total). LSTM-GMM corresponds to BC-RNN in RoboMimicMandlekar et al. (2021), which we reproduced and obtained slightly better results than the original paper. Our results show that Diffusion Policy significantly improves state-of-the-art performance across the board.
Lift Can Square Transport ToolHang Push-T
ph mh ph mh ph mh ph mh ph ph
LSTM-GMM 1.00/0.96 1.00/0.95 1.00/0.88 0.98/0.90 0.82/0.59 0.64/0.38 0.88/0.62 0.44/0.24 0.68/0.49 0.69/0.54
IBC 0.94/0.73 0.39/0.05 0.08/0.01 0.00/0.00 0.03/0.00 0.00/0.00 0.00/0.00 0.00/0.00 0.00/0.00 0.75/0.64
DiffusionPolicy-C 1.00/1.00 1.00/1.00 1.00/0.97 1.00/0.96 0.98/0.92 0.98/0.84 1.00/0.93 0.89/0.69 0.95/0.73 0.91/0.84
DiffusionPolicy-T 1.00/1.00 1.00/0.99 1.00/0.98 1.00/0.98 1.00/0.90 0.94/0.80 0.98/0.81 0.73/0.50 0.76/0.47 0.78/0.66
Table 2: Behavior Cloning Benchmark (Visual Policy) Performance are reported in the same format as in Tab 2. LSTM-GMM numbers were reproduced to get a complete evaluation in addition to the best checkpoint performance reported. Diffusion Policy shows consistent performance improvement, especially for complex tasks like Transport and ToolHang.

5 Evaluation

The benchmark suite spans robomimic manipulation, Push-T, multimodal block pushing, and Franka Kitchen, with both state and visual inputs where available. The tables retain both the best-checkpoint and late-checkpoint-average results. Relative improvements and their aggregation are defined in Appendix B.2.

5.1 Simulation Environments and datasets

Robomimic contributes five tasks, with proficient-human and mixed-proficiency variants. Push-T requires precise contact-rich planar manipulation. Block pushing tests alternative assignments and ordering of two blocks using scripted demonstrations; Kitchen tests variable ordering of four subgoals among seven objects. The prose says Kitchen has 566 demonstrations, while Table 3 lists 656; this edition preserves that inconsistency rather than selecting an unsupported value.

Task # Rob # Obj ActD #PH #MH Steps Img? HiPrec
Simulation Benchmark
Lift 1 1 7 200 300 400 Yes No
Can 1 1 7 200 300 400 Yes No
Square 1 1 7 200 300 400 Yes Yes
Transport 2 3 14 200 300 700 Yes No
ToolHang 1 2 7 200 0 700 Yes Yes
Push-T 1 1 2 200 0 300 Yes Yes
BlockPush 1 2 2 0 0 350 No No
Kitchen 1 7 9 656 0 280 No No
Realworld Benchmark
Push-T 1 1 2 136 0 600 Yes Yes
6DoF Pour 1 liquid 6 90 0 600 Yes No
Peri Spread 1 liquid 6 90 0 600 Yes No
Mug Flip 1 1 7 250 0 600 Yes No
Table 3: Tasks Summary. # Rob: number of robots, #Obj: number of objects, ActD: action dimension, PH: proficient-human demonstration, MH: multi-human demonstration, Steps: max number of rollout steps, HiPrec: whether the task has a high precision requirement. BlockPush uses 1000 episodes of scripted demonstrations.
Original figure
BlockPush Kitchen
p1 p2 p1 p2 p3 p4
LSTM-GMM 0.03 0.01 1.00 0.90 0.74 0.34
IBC 0.01 0.00 0.99 0.87 0.61 0.24
BET 0.96 0.71 0.99 0.93 0.71 0.44
DiffusionPolicy-C 0.36 0.11 1.00 1.00 1.00 0.99
DiffusionPolicy-T 0.99 0.94 1.00 0.99 0.99 0.96
Table 4: Multi-Stage Tasks (State Observation). For PushBlock, p​xpx is the frequency of pushing xx blocks into the targets. For Kitchen, p​xpx is the frequency of interacting with xx or more objects (e.g. bottom burner). Diffusion Policy performs better, especially for difficult metrics such as p​2p2 for Block Pushing and p​4p4 for Kitchen, as demonstrated by our results.

5.2 Evaluation Methodology

Reported late-training performance averages the last ten saved checkpoints across three training seeds. The intended 50 environment initializations were reduced to 22 for robomimic by an evaluation-code bug; the authors state that compared methods used the same protocol. Best-checkpoint averages are also reported where earlier benchmarks used them. State policies train for 4,500 epochs and image policies for 3,000. Diffusion uses position control and baselines use their better-performing velocity controls, so the headline comparison includes those action-space choices.

5.3 Key Findings

The diffusion variants lead the reported comparisons, including short-horizon route choice and long-horizon subgoal ordering. The reported gains are 32% relative on completing both BlockPush targets and 213% on four Kitchen subtasks. Executing eight predicted steps balances consistency and reaction time in most tested tasks. Simulated-latency ablations retain peak performance through four delayed steps; that result is task- and controller-specific.

5.4 Ablation Study

On the robomimic square task, frozen pretrained encoders perform poorly relative to end-to-end training or fine-tuning. Training a vision transformer from scratch reaches 22% success, while fine-tuning its CLIP-pretrained version at a smaller learning rate reaches 98%. ResNet and transformer peak results are close on this task. The limited comparison does not establish a generally best vision architecture or pretraining recipe.

Archicture & From Pretrained
Prertain Datset Scatch frozen finetuning
Resnet18 (in21) 0.94 0.58 0.92
Resnet34 (in21) 0.92 0.40 0.94
ViT-base (clip) 0.22 0.70 0.98
Table 5: Vision Encoder Comparison All models are trained on the robomimic square (ph) task using CNN-based diffusion policy. Each model is trained for 500 epochs and evaluated every 50 epochs under 50 different environment initial conditions.

6 Realworld Evaluation

Four single-arm tasks on two stations test precise pushing, rotations, non-rigid material, pauses, and periodic motion. The convolutional policy with a jointly trained visual encoder performs best among the tested variants. Qualitative perturbation demonstrations are distinct from the standard success-rate evaluations.

Original figure
Human IBC LSTM-GMM Diffusion Policy
Demo pos vel pos vel T-E2E ImgNet R3M E2E
IoU 0.84 0.14 0.19 0.24 0.25 0.53 0.24 0.66 0.80
Succ% 1.00 0.00 0.00 0.20 0.10 0.65 0.15 0.80 0.95
Dur. 20.3 56.3 41.6 47.3 51.7 57.5 55.8 31.7 22.9
Table 6: Realworld Push-T Experiment. a) Hardware setup. b) Illustration of the task. The robot needs to \raisebox{-0.9pt}{1}⃝ precisely push the T-shaped block into the target region, and \raisebox{-0.9pt}{2}⃝ move the end-effector to the end-zone. c) The ground truth end state used to calculate IoU metrics used in this table. Table: Success is defined by the end-state IoU greater than the minimum IoU in the demonstration dataset. Average episode duration presented in seconds. T-E2E stands for end-to-end trained Transformer-based Diffusion Policy
Refer to caption
Figure 7: Realworld Push-T Comparisons. Columns 1-4 show action trajectories based on key events. The last column shows averaged images of the end state. A: Diffusion policy (End2End) achieves more accurate and consistent end states. B: Diffusion Policy (R3M) gets stuck initially but later recovers and finishes the task. C: LSTM-GMM fails to reach the end zone while adjusting the T block, blocking the eval camera view. D: IBC prematurely ends the pushing stage.

6.1 Realworld Push-T Task

Real Push-T requires aligning the block and then moving the end effector to an end zone. Evaluation uses final overlap rather than the maximum reached during the episode. On 20 matched initial conditions, the best diffusion policy succeeds in 95% of trials with mean overlap 0.80, versus human demonstrations at 0.84; the best implicit and recurrent baselines achieve 0% and 20% success. Commands at 10 Hz are interpolated to 125 Hz. A separate demonstration shows recovery from camera occlusion and block displacement, without estimating a perturbation success rate.

Original figure
Figure 8: Robustness Test for Diffusion Policy. Left: A waving hand in front of the camera for 3 seconds causes slight jitter, but the predicted actions still function as expected. Middle: Diffusion Policy immediately corrects shifted block position to the goal state during the pushing stage. Right: Policy immediately aborts heading to the end zone, returning the block to goal state upon detecting block shift. This novel behavior was never demonstrated. Please check the videos in the supplementary material.
Original figure
Human LSTM-GMM Diffusion Policy
Succ % 1.0 0.0 0.9
Figure 9: 6DoF Mug Flipping Task. The robot needs to \raisebox{-0.9pt}{1}⃝ Pickup a randomly placed mug and place it lip down (marked orange). \raisebox{-0.9pt}{2}⃝ Rotate the mug such that its handle is pointing left.
Refer to caption
Pour Spread
IoU Succ Coverage Succ %
Human 0.79 1.00 0.79 1.00
LSTM-GMM 0.06 0.00 0.27 0.00
Diffusion Policy 0.74 0.79 0.77 1.00
Figure 10: Realworld Sauce Manipulation. [Left] 6DoF pouring Task. The robot needs to \raisebox{-0.9pt}{1}⃝ dip the ladle to scoop sauce from the bowl, \raisebox{-0.9pt}{2}⃝ approach the center of the pizza dough, \raisebox{-0.9pt}{3}⃝ pour sauce, and \raisebox{-0.9pt}{4}⃝ lift the ladle to finish the task. [Right] Periodic spreading Task The robot needs to \raisebox{-0.9pt}{1}⃝ approach the center of the sauce with a grasped spoon, \raisebox{-0.9pt}{2}⃝ spread the sauce to cover pizza in a spiral pattern, and \raisebox{-0.9pt}{3}⃝ lift the spoon to finish the task.

6.2 Mug Flipping Task

The policy must turn a mug lip-down and align its handle, combining alternative grasps and pushes. Diffusion succeeds in 18 of 20 trials and qualitatively recovers using repeated pushes or regrasping. The recurrent baseline fails to grasp in all 20 tests, but it was trained on a subset of the diffusion policy’s data; the comparison therefore does not isolate policy architecture alone.

6.3 Sauce Pouring and Spreading

Pouring tests waiting for viscous sauce, centering a ladle, and terminating; spreading combines cyclic coverage with feedback. Diffusion obtains pouring overlap 0.74 versus 0.79 for human demonstrations and spreading coverage 0.77 versus 0.79. The recurrent baseline commonly stays at the bowl or lifts before contacting the sauce. The appendix supplies the evaluation protocol, including the use of the larger coverage estimate from two projected camera views.

7 Realworld Bimanual Tasks

The extension applies the same policy design to two arms. Most additional engineering supports teleoperation and coordinated control rather than task-specific network changes. Results nevertheless vary substantially across skills.

7.1 Observation and Action Spaces

Observations contain actual and desired poses and gripper widths for both arms, plus two scene cameras and two wrist cameras. Actions specify both desired end-effector poses and gripper widths.

7.2 Teleoperation

Two simultaneous SpaceMouse devices were difficult for the operator. The authors instead use two virtual-reality hand controllers or two haptic devices with bilateral position coupling. Haptic feedback is particularly useful for the contact-sensitive egg-beater skill.

7.3 Bimanual Egg Beater

The egg-beater policy succeeds in 55% of 20 trials after 210 demonstrations. Failures include unfamiliar initial placement, missed handle grasps, and losing the crank handle. In a separate small teleoperation comparison, the same expert completes 0 of 10 demonstrations without haptic feedback and 10 of 10 with it. This illustrates demonstration-collection difficulty rather than a randomized study of operators.

Refer to caption
Figure 11: Bimanual Egg Beater Manipulation. The robot needs to \raisebox{-0.9pt}{1}⃝ push the bowl into position (only if too close to the left arm), \raisebox{-0.9pt}{2}⃝ approach and pick up the egg beater with the right arm, \raisebox{-0.9pt}{3}⃝ place the egg beater in the bowl, \raisebox{-0.9pt}{4}⃝ approach and grasp the egg beater crank handle, and \raisebox{-0.9pt}{5}⃝ turn the crank handle 3 or more times.

7.4 Bimanual Mat Unrolling

After 162 demonstrations, mat unrolling succeeds in 75% of 20 trials, choosing leftward or rightward unrolling from the initial state. Missed initial grasps often lead to repeated unsuccessful motion instead of recovery. The appendix retains every evaluation’s initial and final state.

Refer to caption
Figure 12: Bimanual Mat Unrolling. The robot needs to \raisebox{-0.9pt}{1}⃝ pick up one side of the mat (if needed), using the left or right arm, \raisebox{-0.9pt}{2}⃝ lift and unroll the mat (if needed), \raisebox{-0.9pt}{3}⃝ ensure that both sides of the mat are grasped, \raisebox{-0.9pt}{4}⃝ lift the mat, \raisebox{-0.9pt}{5}⃝ place the mat oriented with the table, mostly centered, and \raisebox{-0.9pt}{6}⃝ release the mat.

7.5 Bimanual Shirt Folding.

Shirt folding succeeds in 75% of 20 trials after 284 demonstrations. The multi-step sequence requires close coordination and collision avoidance in the low-level controller. Failures involve missed sleeve or collar grasps and continued adjustment after folding. The initial and final rollout panels remain available for inspecting these outcomes.

Refer to caption
Figure 13: Bimanual Shirt Folding. The robot needs to \raisebox{-0.9pt}{1}⃝ approach and grasp the closest sleeve with both arms, \raisebox{-0.9pt}{2}⃝ fold the sleeve and release, \raisebox{-0.9pt}{3}⃝ drag the shirt closer (if needed), \raisebox{-0.9pt}{4}⃝ approach and grasp the other sleeve with both arms, \raisebox{-0.9pt}{5}⃝ fold the sleeve and release, \raisebox{-0.9pt}{6}⃝ drag the shirt to a orientation for folding, \raisebox{-0.9pt}{7}⃝ grasp and fold the shirt in half by its collar, \raisebox{-0.9pt}{8}⃝ drag the shirt to the center, and \raisebox{-0.9pt}{9}⃝ smooth out the shirt and move the arms away.

8 Related Work

Explicit policies predict actions directly, using regression, discretization, or mixtures to represent alternatives. Implicit policies optimize a learned energy, gaining distributional flexibility but facing normalization and sampling difficulties. Diffusion contributes an alternative distributional policy with high-dimensional sequence prediction. Compared with diffusion planning and reinforcement-learning work, this paper focuses on behavior-cloned visuomotor control and the action-space, feedback, and latency choices needed on physical robots.

9 Limitations and Future Work

The method inherits behavior cloning’s dependence on adequate demonstrations and does not automatically learn from unsuccessful or suboptimal behavior. Iterative denoising also costs more compute and adds latency relative to simple direct policies. Action sequences partially mask that latency but may be insufficient for fast feedback tasks. Reinforcement learning and accelerated diffusion sampling are proposed directions, not capabilities established here.

10 Conclusion

The evidence supports diffusion as an effective policy representation for these manipulation datasets and hardware setups. Its performance depends on the complete control design: action chunks, position control, visual conditioning, and execution feedback. Demonstration quality, robot capabilities, model architecture, and pretraining remain important; distributional expressivity alone does not remove those constraints.

11 Acknowledgement

The authors acknowledge colleagues and hardware support, and funding from Toyota Research Institute and the US National Science Foundation. The conclusions are the authors’ rather than official sponsor positions.

Appendix A Diffusion Policy Implementation Details

The implementation details below retain the original normalization, rotation, augmentation, and hyperparameter choices needed to reproduce the experiments.

A.1 Normalization

Scale each non-rotational action dimension to the interval [-1, 1], matching clipping during denoising. Standardizing by mean and variance can otherwise make some actions unreachable. Near-constant dimensions are centered without scaling to avoid numerical problems; rotation representations are left unchanged.

A.2 Rotation Representation

Velocity actions use three-dimensional axis-angle rotations, with small commands reducing exposure to discontinuities. Position-controlled tasks use the continuous six-dimensional rotation representation. [70]

A.3 Image Augmentation

Training uses random image crops; inference uses a fixed center crop of the same size. The preserved hyperparameter tables specify each task’s image and crop resolution.

A.4 Hyperparameters

Convolutional hyperparameters transfer more consistently across tasks; transformer dropout, weight decay, and size require more care. Push-T is an exception to the usual FiLM conditioning rule: its reported convolutional simulation result uses inpainting. Simulation uses 100 denoising steps at both training and inference; this appendix specifies 16 inference steps for real-world DDIM, versus 10 in Section 3.4. State and image batch sizes are 256 and 64, with cosine learning-rate decay and warmup.

H-Param Ctrl To Ta Tp ImgRes CropRes #D-Params #V-Params Lr WDecay D-Iters Train D-Iters Eval
Lift Pos 2 8 16 2x84x84 2x76x76 256 22 1e-4 1e-6 100 100
Can Pos 2 8 16 2x84x84 2x76x76 256 22 1e-4 1e-6 100 100
Square Pos 2 8 16 2x84x84 2x76x76 256 22 1e-4 1e-6 100 100
Transport Pos 2 8 16 4x84x85 4x76x76 264 45 1e-4 1e-6 100 100
ToolHang Pos 2 8 16 2x240x240 2x216x216 256 22 1e-4 1e-6 100 100
Push-T Pos 2 8 16 1x96x96 1x84x84 256 22 1e-4 1e-6 100 100
Block Push Pos 3 1 12 N/A N/A 256 0 1e-4 1e-6 100 100
Kitchen Pos 2 8 16 N/A N/A 256 0 1e-4 1e-6 100 100
Real Push-T Pos 2 6 16 2x320x240 2x288x216 67 22 1e-4 1e-6 100 16
Real Pour Pos 2 8 16 2x320x240 2x288x216 67 22 1e-4 1e-6 100 16
Real Spread Pos 2 8 16 2x320x240 2x288x216 67 22 1e-4 1e-6 100 16
Real Mug Flip Pos 2 8 16 2x320x240 2x288x216 67 22 1e-4 1e-6 100 16
Table 7: Hyperparameters for CNN-based Diffusion Policy Ctrl: position or velocity control To: observation horizon Ta: action horizon Tp: action prediction horizon ImgRes: environment observation resolution (Camera views x W x H) CropRes: random crop resolution #D-Params: diffusion network number of parameters in millions #V-Params: vision encoder number of parameters in millions Lr: learining rate WDecay: weight decay D-Iters Train: number of training diffusion iterations D-Iters Eval: number of inference diffusion iterations (enabled by DDIM Song et al. (2021))
H-Param Ctrl To Ta Tp #D-params #V-params #Layers Emb Dim Attn Drp Lr WDecay D-Iters Train D-Iters Eval
Lift Pos 2 8 10 9 22 8 256 0.3 1e-4 1e-3 100 100
Can Pos 2 8 10 9 22 8 256 0.3 1e-4 1e-3 100 100
Square Pos 2 8 10 9 22 8 256 0.3 1e-4 1e-3 100 100
Transport Pos 2 8 10 9 45 8 256 0.3 1e-4 1e-3 100 100
ToolHang Pos 2 8 10 9 22 8 256 0.3 1e-4 1e-3 100 100
Push-T Pos 2 8 16 9 22 8 256 0.01 1e-4 1e-1 100 100
Block Push Vel 3 1 5 9 0 8 256 0.3 1e-4 1e-3 100 100
Kitchen Pos 4 8 16 80 0 8 768 0.1 1e-4 1e-3 100 100
Real Push-T Pos 2 6 16 80 22 8 768 0.3 1e-4 1e-3 100 16
Table 8: Hyperparameters for Transformer-based Diffusion Policy Ctrl: position or velocity control To: observation horizon Ta: action horizon Tp: action prediction horizon #D-Params: diffusion network number of parameters in millions #V-Params: vision encoder number of parameters in millions Emb Dim: transformer token embedding dimension Attn Drp: transformer attention dropout probability Lr: learining rate WDecay: weight decay (for transformer only) D-Iters Train: number of training diffusion iterations D-Iters Eval: number of inference diffusion iterations (enabled by DDIM Song et al. (2021))
Original figure
Figure 14: Observation Horizon Ablation Study. State-based Diffusion Policy is not sensitive to observation horizon. Vision-based Diffusion Policy prefers low but >1>1 observation horizon, with 22 being a good compromise for most tasks.
Original figure
Figure 15: Data Efficiency Ablation Study. Diffusion Policy outperforms LSTM-GMM Mandlekar et al. (2021) at every training dataset size.

A.5 Data Efficiency

The reported data-size ablation favors Diffusion Policy over the recurrent Gaussian-mixture baseline at every tested training-set size. It does not establish performance without demonstrations or outside the evaluated data distributions.

Appendix B Additional Ablation Results

The additional ablations clarify the role of observation history and the exact calculation behind the headline performance gain.

B.1 Observation Horizon

State-based policies are relatively insensitive to the tested observation horizons. Longer image histories can reduce performance, especially for the convolutional model. A two-step observation horizon works well across most of these experiments.

B.2 Performance Improvement Calculation

For each included task, select the best baseline and the better diffusion backbone, then compute (diffusion - baseline) / baseline. Average these taskwise relative gains, excluding the mixed-human variants, to obtain 0.46858, rounded to 46.9%. This is a best-variant, equally weighted relative improvement; it is neither a pooled success-rate difference nor the gain of one fixed policy configuration.

Appendix C Realworld Task Details

The real-world protocols specify demonstration counts, checkpoint selection, initial-condition matching, and termination rules.

C.1 Push-T

The Push-T protocol evaluates final block alignment together with completion of the end-zone movement.

C.1.1 Demonstrations

A practiced operator records 136 demonstrations, varying initial states by pushing or tossing the block. These demonstrations represent a proficient operator rather than a distribution of novice behavior.

C.1.2 Evaluation

Each method trains for 12 hours. Evaluation normally uses the last checkpoint; implicit behavior cloning instead uses the checkpoint with minimum training action error. Each method starts from the same 20 initial conditions, aligned using top-camera overlays. Episodes end after 0.5 seconds in the end zone or at 60 seconds, and overlap is measured in top-camera pixels.

C.2 Sauce Pouring and Spreading

Sauce-task evaluation varies the dough and bowl positions and measures overlap or coverage after policy termination.

C.2.1 Demonstrations

The appendix states 50 demonstrations per sauce task with 90% used for training; Table 3 instead lists 90 proficient demonstrations for each. Both source values remain visible. Initial placements and sauce shapes vary, and some rotational teleoperation commands are discarded to avoid unintended scooping or spilling.

C.2.2 Evaluation

Diffusion and recurrent policies train for 1,000 epochs and use their final checkpoints. Evaluation matches random object positions, but not the exact initial sauce shape, while holding sauce quantity fixed. Episodes end after lifting away for 0.5 seconds or an operator safety stop. Each camera image is projected onto the table, and the larger of the two coverage estimates is reported.

Appendix D Realworld Setup Details

The low-level robot controllers enforce limits and interpolate commands; those safeguards are part of the experimental system rather than learned guarantees of the policy.

D.0.1 UR5 robot station

The UR5 accepts position targets at 125 Hz, interpolated from 10 Hz human or policy commands. The controller limits speed to 0.43 m/s and confines the end effector near the table plane. Two of five cameras provide policy images, downsampled to 320 by 240 at 10 Hz; demonstrations use a SpaceMouse.

D.1 Franka Robot Station

The Franka station uses a constrained differential-kinematics controller to convert desired end-effector poses to joint motion. It can impose collision, joint, and workspace limits. Haptic teleoperation uses a separate torque controller, while policy inference uses the non-haptic controller. The source describes a 200 Hz haptic optimization and roughly 1 kHz mid-level control, with 10 Hz policy commands and interpolated targets. Camera images are downsampled to 320 by 240.

D.2 Initial and Final States of Bimanual Tasks

All 20 initial and final states are retained for each of the three bimanual tasks. Green and red outlines denote successful and failed rollouts. Mat and shirt images use planar projections for visibility. The source’s introductory sentence says four tasks, but the displayed panels cover mat unrolling, egg beating, and shirt folding.

Refer to caption
Figure 16: Initial states for Mat Unrolling
Refer to caption
Figure 17: Final states for Mat Unrolling
Refer to caption
Figure 18: Initial states for Egg Beater
Refer to caption
Figure 19: Final states for Egg Beater
Refer to caption
Figure 20: Initial states for Shirt Folding
Refer to caption
Figure 21: Final states for Shirt Folding
Concept demonstration

Open image ↗