Overview
Long observation histories can work for single-task Diffusion Policies when architecture, data, and task demands are matched. Variable-history training improves several limited-data cases.
At a glance
The study varies context length, demonstration count, manipulation stability, and denoising architecture. A UNet with cross-attention often tolerates naive history scaling better than the tested FiLM and transformer alternatives. Training jointly over multiple context lengths further improves selected low-data regimes. Results span five simulated tasks and one small hardware study; each reported configuration uses one training run, and simulated performance selects the best tested checkpoint.
1 Introduction
Short histories can hide task phase or past decisions, but failures of longer histories need not establish that naive scaling is fundamentally impossible. The paper systematically tests architecture, data scale, and manipulation difficulty before introducing a variable-history curriculum. Its scope is recent, continuous image history for single-task policies trained from scratch, rather than distant symbolic memory or a general pretrained robot foundation model.
1.1 Statement of Contributions
Nearly two hundred trained policies probe five simulated tasks, three dataset sizes, multiple context lengths, and alternative architectures. The proposed curriculum reports roughly 1.25–2-fold improvement over the strongest naive-scaling baseline in selected low-data settings while preserving performance in other tested regimes. A separate analysis revisits whether past-action prediction or freezing the visual encoder explains earlier long-context improvements. These findings are empirical within the tested training setup.
1.2 Related Works
Past-token prediction, experience retrieval, and key-frame selection address related memory problems with different assumptions. The present study tests whether direct recent-history scaling can work before compression/filtering. It also distinguishes pretrained or multi-task memory models from single-task learning from scratch. Prior causal-confusion explanations are relevant hypotheses, not established mechanisms for all diffusion policies.
2 Preliminaries
A diffusion policy conditions action denoising on image/state history. Both the length of observed history and the amount of past action included in the prediction target affect the experiments.
2.1 Diffusion Policy
The policy models a chunk containing past and future actions conditioned on the last Tₒ observations. Fixed-history baselines predict the full past window, following Diffusion Policy code. Simulation contexts reach eighty steps; the hardware test uses ninety-two. These are continuous recent observations, not arbitrary long-term episode retrieval.
Equation 1
| (1) |
2.2 Diffusion Policy Architecture
An image encoder first embeds observations; a denoiser then combines these embeddings, proprioception, and diffusion time to generate actions. The study compares a UNet with global FiLM conditioning, a UNet with token-wise cross-attention, and a transformer denoiser. The image encoders are normally trained from random initialization, so conclusions about backbone preference should not be transferred directly to large pretrained DiTs.
3 Evaluating Long-Context Diffusion Policies
Push-T, Square, and Lift are solvable with short histories; Push-and-Return and Grasp-and-Return additionally require remembering an initial location. Each simulated task uses half, usual, and twice its task-specific demonstration count. The main sweep uses UNet plus cross-attention, with separate architecture comparisons. Context lengths range from one to twenty for short-history tasks, up to eighty for pushing return, and forty-eight for grasping return.
3.1 Benchmark Tasks Requiring Long Context Lengths
In the two return tasks, an object starts on one side, moves to the center, the robot visits a neutral configuration, and the object must return to its original side. Pushing is locally unstable; grasping can tolerate larger control errors. Task success requires the correct side. Manipulation completion accepts either side, and contextual success is the fraction of completed manipulations that choose correctly.
The physical marshmallow task requires exactly two transfers before signaling completion. The target bowl starts with varying contents, preventing final quantity alone from revealing how many transfers occurred. Contextual success in the return tasks is conditional on manipulation completion and is not an unconditional memory score over all failed rollouts.
3.2 Empirical Results
Lift remains strong as history increases at all tested data scales. Push-T and Square lose success at longer contexts with limited data, but the gap shrinks as demonstrations increase. Grasp-and-Return learns useful long-context behavior even with twelve demonstrations, while Push-and-Return needs more data. The pattern is consistent with stable grasping tolerating imperfect learned control. It does not imply that every task benefits from more context or that data alone resolves missing task-state information.
3.3 Additional Comments
At long contexts, manipulation completion often tracks total success, while successful completions usually return to the correct side. This suggests that manipulation learning is a major bottleneck in these settings. Because contextual success conditions on completion, it cannot establish that memory is correct on every failed rollout. Short-history Grasp-and-Return also fails manipulation itself, plausibly because ambiguous task phase causes switching and out-of-distribution motions.
4 Training Long-Context Diffusion Policies
The training analysis first changes conditioning architecture, then jointly trains across histories, and finally separates past-action prediction from freezing a pretrained short-history observation encoder.
4.1 Cross-Attention Conditioning
UNet cross-attention generally improves long-context behavior in limited data relative to FiLM, while short-context differences are smaller. The tested DiT performs poorly on several tasks even after increasing its size, removing causal attention within action chunks, and adding denoising steps. This is evidence about these single-task, scratch-trained configurations; it is not a general rejection of transformer policies or pretrained diffusion backbones.
4.2 Variable History Training
Variable-history training samples a context length for each training iteration and clips the predicted past-action window to the available history. Random Sprinkle uses the maximum history with probability 0.8 and otherwise samples shorter lengths uniformly. Progressive training starts at the minimum history, expands the available lengths through the first half of training, then switches to Random Sprinkle.
Progressive training with a short past-action target works best when naive long-context learning struggles. Where naive scaling already works, Random Sprinkle with full past-action prediction is often preferable. Thus the best recipe depends on the effective data/task regime; the experiments do not establish a single universally optimal curriculum. The original algorithm is retained with its full loss notation and loop structure.
4.3 Revisiting Past-Token Prediction
All naive baselines already predict past actions, yet some fail at long contexts. Freezing an observation encoder learned by a short-context policy also contributes to success, sometimes together with past-action prediction, with Push-and-Return an exception. This analysis uses the paper’s UNet/cross-attention implementation rather than an exact reproduction of the earlier DiT system, and omits the earlier inference-time verification step from the main comparisons.
5 Limitations and Discussion
Long histories increase inference overhead because many images must be encoded in the closed loop. Distillation and adaptive context selection at inference are proposed future directions, not implemented speedups. Further limits include single training seeds per configuration, task-dependent recipe selection, and a small hardware comparison on a modified setup. Rollout confidence intervals do not measure uncertainty from retraining or the best-checkpoint selection procedure.
6 Conclusion
Naive history scaling can be effective given sufficient task data and an appropriate single-task architecture. Cross-attention conditioning and variable-history training reduce failures in several limited-data cases. Encoder freezing complicates attributing prior gains solely to past-action prediction. The evidence supports carefully matched baselines and data sweeps, while leaving broader tasks, repeated training seeds, and inference-efficient memory unresolved.
Acknowledgments
Support includes NSF HAND and graduate fellowship funding, Canadian graduate support, Amazon, and shared MIT/Texas A&M computing resources. The original acknowledgment retains institutional and grant details; funders’ support does not imply endorsement of the findings.
Appendix A Additional Related Works
Additional related work distinguishes recent-history policy learning from language-based retrieval, hierarchical memory, dynamic control, and long-context language-model evaluation.
A.1 Context Length in Imitation Learning
Key-frame filtering and language summaries compress history but may rely on pretrained models or annotation. The cited multi-scale memory model separately encodes recent video and longer summaries. Joint action/video prediction offers another route but adds data and compute requirements. These approaches address overlapping memory needs with resources and interfaces different from the current scratch-trained single-task policies.
A.2 Context Length in Reinforcement Learning in Robotics
Dynamic-control reinforcement learning often uses longer joint-state histories than vision-based imitation. Some methods combine an encoded full history with direct recent state; others use a slower adaptation module and a faster control policy. Comparisons between recurrent and finite-history models have mixed task-dependent outcomes, so context design should follow observation and dynamics requirements.
A.3 Context Length Understanding in LLMs
Language-model studies show sensitivity to where relevant information appears, difficult long dependencies, and attention to distant tokens. These motivate more careful robotics memory benchmarks, but do not themselves establish the cause of robot-policy failures.
Appendix B Diffusion Policy Details
The appendix specifies diffusion loss, conditioning architectures, checkpoint selection, parameter counts, and task-dependent horizons.
B.1 Overview
A shared-per-camera ResNet18 embeds each timestep, normally from random initialization. Image/state tokens condition prediction of the Gaussian noise added to action chunks. The retained denoising objective includes the diffusion schedule and expectation over data, diffusion time, and noise. Encoder weights are shared through time, avoiding a separate visual network for each history position.
Equation 2
| (2) |
B.2 Comparison of Diffusion Policy Architectures
FiLM concatenates all camera/time embeddings into one vector and maps it to scale/bias terms for UNet layers. Cross-attention preserves time-indexed tokens, concatenates cameras and state within each step, adds positional encoding, and lets intermediate UNet features attend to them. DiT uses transformer decoder layers instead of convolutional UNet layers.
The DiT baseline is enlarged to a comparable parameter scale, removes causal masking within the denoised action chunk to match UNet information flow, and uses more sampling steps. Its continued weakness is therefore not explained solely by the original much smaller model, but the study still covers particular scratch-trained architectures and hyperparameters.
B.3 Training Details and Hyperparameters
Training budgets are matched across compared context lengths within each task. Checkpoint and architecture details matter when interpreting the performance curves.
B.3.1 Checkpoint Selection
For simulated tasks, candidates include the five best validation-loss checkpoints, five best action-matching checkpoints, and three late evenly spaced checkpoints; some configurations retain only the latest late checkpoint. Every candidate is rolled out and the highest success is reported. Hardware uses the latest checkpoint. This evaluation-driven selection can overestimate a separately selected deployment model’s success.
Push-T trains for 150,000 steps, Grasp-and-Return for 100,000, and other tasks for 200,000, using initial learning rate 10⁻⁴ with cosine decay. Each result is one training run. The authors explicitly do not estimate variability due to random training seeds; earlier low-data checkpoints sometimes work better, consistent with overfitting.
B.3.2 Hyperparameters
FiLM total parameter count grows from about 89 million at two frames to 235 million at eighty, whereas cross-attention stays near 151 million and the enlarged DiT near 131 million. Encoder parameters are about 22.4 million. Action denoising lengths are rounded to multiples of four while retaining roughly sixteen future actions and the chosen past window. Table labels and the prose use horizon notation differently; the original configurations and values remain intact rather than silently standardizing them.
| UNet+FiLM | UNet+xAttn | DiT | |
| Denoiser core | |||
| Conditioning mechanism | FiLM (global) | Cross-attention (token seq.) | Cross-attention (token seq.) |
| UNet Down channels | – | ||
| Conv kernel size | 5 | 5 | – |
| GroupNorm groups | 8 | 8 | – |
| Approximate parameter counts | |||
| Denoiser parameters | at to at | ||
| Encoder parameters | |||
| Total parameters | at to at | ||
| Setting | Push-T | Push-and- | Grasp-and- | Marshmallows | Square | Lift |
|---|---|---|---|---|---|---|
| return | return | (hardware) | (robomimic) | (robomimic) | ||
| Image input () | ||||||
| Crop () | ||||||
| Cameras | overhead, wrist | overhead, wrist | overhead, wrist | overhead, wrist | agentview, eye-in-hand | agentview, eye-in-hand |
| Proprioception dim | 3 | 2 | 13 | 13 | 9 | 9 |
| Action dim | 2 | 2 | 13 | 13 | 10 | 10 |
| Context length swept () | 1, 2, 5, 10, 12, 16, 20 | 4, 8, 32, 60, 72, 80 | 4, 8, 16, 20, 24, 48 | 92 | 1, 2, 5, 10, 12, 16, 20 | 1, 2, 5, 10, 12, 16, 20 |
| Future chunk length | +14 to +16 | to | to | |||
| Data (low / usual / high) | 80 / 160 / 320 | 24 / 48 / 96 | 12 / 24 / 48 | 100 | 50 / 100 / 200 | 25 / 50 / 100 |
| Total training steps |
Appendix C Task Details
“Usual” data scale is task-specific, not a common demonstration count. The appendix gives collection procedures and success criteria.
C.1 Task and Data Collection Descriptions
Usual demonstration counts are 160 Push-T, 100 Square, fifty Lift, forty-eight Push-and-Return, and twenty-four Grasp-and-Return. Pushing trajectories use teleoperation, with mirrored return demonstrations enforcing symmetry between the two initial modes. Grasping return data come from a heuristic simulation policy. Hardware uses one hundred VR demonstrations, an overhead and wrist camera, and six-dimensional end-effector pose control. The return experiments only test two destination modes; broader combinatorial memory remains future work.
C.2 Evaluation Details
Simulated success uses two hundred trials, with 95% Wilson intervals for binary outcomes. Return tasks use one hundred trials per initial side. Pushing has a fifty-second limit and requires over 95% area overlap at the target; grasping uses contact with the center region and return to the originating mat. Hardware evaluates twenty trials with varying initial target-bowl contents. These intervals describe rollout sampling for a trained model, not uncertainty from training runs or checkpoint selection.
Appendix D Additional Discussion
Additional analyses examine prior benchmark choices, tolerance to grasp error, open-loop versus closed-loop losses, and observability of the demonstrator’s task phase.
D.1 Reasons for Prior Works’ Criticism of Naive History Scaling
The authors attribute overly broad criticism of naive history scaling to inadequate dataset sweeps, weak conditioning/backbone choices, and too narrow a task selection. Their data support testing both memory-requiring and short-history-solvable tasks at several data scales. This is an alternative explanation for observed failures, not proof that prior methods are unnecessary in every setting.
D.2 Impact of Grasp as Manipulation Primitive
Expert grasp contacts lie on the brick’s central axis, while learned long-context policies deviate by mean absolute offsets of 1.2, 1.4, and 1.3 cm at histories twenty, twenty-four, and forty-eight; medians are one centimetre. Successful grasping despite these errors supports the local-stability hypothesis. It does not independently vary all task differences between grasping and pushing.
| Context Length | 20 | 24 | 48 |
|---|---|---|---|
| Mean | 0.012 | 0.014 | 0.013 |
| Median | 0.010 | 0.010 | 0.010 |
D.3 Reasons for Long-Context Failure
Long-context failures persist in several low-data regimes. The diagnostics separate supervised prediction quality from closed-loop control and correct return decisions.
D.3.1 Training and test loss
On low-data Square with FiLM, longer-history checkpoints can have lower training and validation loss yet lower rollout success. Better fit on expert-distribution samples does not guarantee stable closed-loop behavior, consistent with overfitting and covariate shift. The authors emphasize that this plot uses one training run and may be noisy; a multi-seed replication is needed to establish robust cluster trends.
D.3.2 Manipulation Skill Learning
In low-data Push-and-Return, both total success and manipulation completion decline with long histories, while the conditional correct-return rate remains high among completions. The authors infer that learning the manipulation skill is the principal bottleneck for these runs. Conditioning on completion limits that diagnosis: it does not directly test the internal memory of every failed rollout.
D.4 When are short-context lengths sufficient?
Short histories can fail even the relaxed manipulation criterion when task phase is visually ambiguous. Grasp-and-Return may alternate between going to center and returning to neutral, causing jitter and collisions. In planar pushing, the pusher’s side relative to the block more directly reveals motion direction. Thus observability depends on the manipulation primitive; more demonstrations alone cannot always replace missing history or explicit phase guidance.
Appendix E Additional Experiments
Further experiments test capacity matching, curriculum choices, past-action prediction, encoder freezing, inference-time verification, and the physical memory task.
E.1 Cross-Attention Conditioning
Cross-attention’s advantage over FiLM at long histories shrinks with more data. A second comparison adjusts FiLM UNet size to match total cross-attention parameters, adding capacity at twenty frames and removing it at eighty; the low-data conclusions remain similar. This reduces a simple parameter-count explanation while leaving the tested architectural/training scope unchanged.

E.2 Variable History Training Ablations
Progressive training with short past targets helps when naive scaling fails, particularly low-data Push-T, Square, and Push-and-Return. When naive scaling already succeeds—including Grasp-and-Return even with twelve demonstrations—Random Sprinkle with full past prediction performs best. The appendix therefore recommends choosing based on observed scaling behavior, a qualification to the main text’s suggestion of use without dataset-size benchmarking.
E.3 Past-Token Prediction
Past-action prediction alone yields inconsistent gains: clearer improvements appear for some usual-data long-context tasks, but short-context and limited-data trends are less consistent. Encoder freezing and past-action prediction can reinforce one another. Variable-history comparisons select the better curriculum hyperparameter, so they include a tuning advantage that should be recognized.
The earlier inference-time verifier samples several action chunks and favors agreement with executed past actions. It is excluded from main comparisons for speed. A small check gives Square 176/200 versus 175/200 without verification and Push-T 190/200 versus 188/200, differences the authors find statistically insignificant. This limited check does not rule out benefits on other tasks.
| Task | With Verification | W/O Verification |
|---|---|---|
| Square | 176/200 | 175/200 |
| Push-T | 190/200 | 188/200 |
E.4 Hardware Validation
With ninety-two-frame context and one hundred demonstrations, cross-attention succeeds in 18/20 marshmallow trials, versus 14/20 each for FiLM and DiT. Confidence intervals overlap, and all models use one training run. The setup and embodiment differ from the earlier 250-demonstration study, so the result is not a controlled head-to-head reproduction. Qualitative observations favor cross-attention’s smoother trajectories and bowl clearance, while DiT sometimes regrasps unnecessarily; these motion-quality claims are not quantified by a separate metric.