Overview
At a glance
Gated Memory Policy (GMP) lets a robot decide when historical observations and actions are useful. A calibrated binary gate activates cross-attention to cached history; diffusion noise on past actions reduces overfitting. The authors report a 30.1% average success-rate improvement over long-history baselines on MemMimic and 26.6% over vision-language-action baselines on MIKASA-Robo, while retaining competitive performance on largely memory-free RoboMimic tasks. Memory remains bounded by a finite buffer.
1 Introduction
Memory demands vary: ordinary pick-and-place may need only the current observation, returning an object can require remembering earlier events in the same trial, and adjusting a throw may require learning from previous attempts. Simply feeding longer histories to a policy can increase overfitting, distribution shift, and latency.
GMP separates when to recall from what to retrieve. It learns retrieval directly from visual and action histories and calibrates a gate on held-out demonstrations. MemMimic evaluates both within-trial recall and adaptation across repeated attempts, including simulation and physical robots.
Related references: [30], [26], [5], [9], [20], [14], [6], [43].
2 Related Work
Earlier robot memory systems use hand-designed traces, trajectories, textual plans, or retrieval databases. Longer-context neural policies can learn memory implicitly, but often add prediction tasks or retain irrelevant history. GMP uses a lightweight cross-attention module over visual and action tokens, with a separate gate to suppress memory when it hurts generalization.
Related references: [12], [6], [10], [49], [21], [29], [1], [30], [13], [5], [7], [3], [28], [19], [18], [15], [27], [47], [36], [25], [42], [39], [16], [4], [38], [43], [22], [23].
3 Method
3.1 Background: Transformer-based Diffusion Policy
The backbone is a Transformer diffusion policy: current images and proprioception condition denoising of a future action chunk. Training minimizes action reconstruction error; DDIM performs iterative denoising at deployment. Extending full self-attention to every historical token increases computation quadratically and can amplify distribution shift.
Related references: [7], [28], [37].
Equation 1
| (1) |
3.2 Gated Memory Policy
An MLP encodes historical action chunks. A pretrained ViT and multi-head attention pooling compress each sampled image into one token. Cached image and action tokens form a sliding history window queried by cross-attention; the history need not repeatedly attend to itself.
Historical actions receive diffusion noise during both training and inference. At each denoising step they are one step cleaner than the future actions, providing informative context without relying on unrealistically clean history.
A gate computed from the current image and proprioception either adds the history-attention output to the residual stream or skips it. Calibration first trains memory-on and memory-off policies, compares their action errors on validation trajectories, and labels states where memory substantially helps. The learned gate is then frozen while the final policy is trained on all demonstrations. End-to-end gate training instead tends to overuse history unless strongly regularized, which can suppress needed recall.
Related references: [11], [44], [46], [32], [4].
4 Experiments
MemMimic tests remembering object locations or colors and adapting pushing, flinging, and casting over successive trials. RoboMimic checks whether memory harms Markovian manipulation; MIKASA-Robo adds memory-intensive tasks and comparisons with vision-language-action policies. Controlled diffusion-policy, past-token-prediction, and recurrent baselines share the visual encoder.
Longer history alone hurts several Markovian tasks and does not reliably solve harder physical or cross-trial tasks. GMP attends to earlier informative views and to overshoot or undershoot outcomes, while often ignoring history: the gate is off for 73% of Match Color and 58% of Iterative Pushing steps. Attention visualizations support selective retrieval, but are not a separate causal test.
Validation-based gate calibration avoids the overuse/underuse tradeoff of joint gate training. Cross-attention scales linearly with history length, and skipping it reduces inference cost further. Noise ablations favor the one-step-cleaner history schedule over clean histories, arbitrary noise levels, or a train/test noise mismatch. Results and original benchmark plots follow.
Related references: [30], [5], [36], [19], [31], [35], [7], [43], [44], [2], [45], [4].
5 Limitations and Future Work
The fixed attention window can discard information needed much later. Selective token replacement or higher-level memory could extend the horizon; these are future directions. Results concern the tested tasks and demonstration distributions, rather than unrestricted lifelong adaptation.
6 Conclusion
Selective recall, efficient cached retrieval, and noise-augmented action histories together improve memory-dependent manipulation without forcing memory into every control decision.
7 Acknowledgments
The authors acknowledge collaborators and funding from Stanford, the Toyota Research Institute, and other supporting organizations; full credits remain in the original paper.
8 Supplementary Materials
8.1 How Long History Can GMP Recall?
In a delayed Match Color variant, the policy recalls a cue after waits of 5–600 seconds. With 6,000 image frames and 6,000 actions, the authors report 99.0% ± 1.0% success and 0.16-second inference for eight denoising steps on an RTX 5090. This experiment freezes the visual encoder and trains on cached features; it does not establish unbounded memory.
Related references: [43].
8.2 Overlapped Trajectory Training
Overlapping trajectory batches share most observations. Encoding each unique image once per batch and reusing its features across subsequences reduces training memory while still updating the visual encoder end to end. Features are recomputed after each optimization step.
Related references: [43], [46].
Equation
8.3 MemMimic Details
MemMimic distinguishes memory of a single execution from adaptation across attempts. Training usually uses SigLIP2-B/16 vision features and chunked absolute or relative pose actions; timing and observation conventions vary with each environment.
Match Color and Discrete Place Back require remembering a cube’s origin after a pause. Short-history policies can remain stuck in the pause or choose a random destination; sufficiently long history solves these simpler tasks. Continuous Place Back uses 262 real demonstrations and random action intervals to tolerate timing variation. Its evaluation covers 40 initial states.
Flip and Place Back uses 2,166 trajectories, 12 training cups, and more than 30 scenes. Deployment tests unseen settings and cups. The gate remains always on for this task because the ungated memory policy already works; surface-height errors and difficult geometry still cause failures.
Iterative Pushing uses 500 simulated episodes with hidden friction and six attempts. Demonstrations adjust velocity by binary search, and GMP uses a longer buffer spanning two trials. Iterative Flinging also uses 500 episodes with variable cloth mass; its success criterion is relaxed because the cloth simulator can be unstable. These adaptation behaviors are learned from demonstrated trial-and-error procedures.
Iterative Casting uses 80 real training episodes across two friction surfaces and three attempts per episode. Timing and camera latency are randomized during training. Twenty evaluation episodes test both surfaces. Baselines can collapse to an intermediate velocity or fail to maintain a corrected velocity across trials; GMP uses previous outcomes to adjust.
Related references: [24], [41], [8], [48], [33], [40].
8.4 RoboMimic Evaluation Details
RoboMimic evaluation covers Square, Transport, and Tool Hang. Policies train for 400 epochs; checkpoints every 20 epochs are each tested on 100 episodes, and the best success rate is reported.
Related references: [30].
8.5 MIKASA-Robo Evaluation Details
Five MIKASA-Robo tasks use 250 demonstrations per task from a state-based PPO oracle. Policies train for 500 epochs, and the best checkpoint and action-horizon configuration are reported. Converting noisy joint deltas to absolute joint actions improves supervision. The small training set particularly limits RememberColor9; these checkpoint-selection and data constraints matter when comparing results.
Related references: [5].
8.6 Baseline Details
Baselines vary both history and prediction objectives: no-history diffusion policy, 16-step and 120-step histories, past-token prediction, and a recurrent behavioral-cloning policy. GMP normally retrieves 120 actions and 15 sampled images, with longer buffers for iterative tasks. Real-world long-history baselines sometimes omit proprioception because noisy histories produce severe jitter.
8.7 Visualization of the Memory Gate Calibration
Calibration averages repeated validation predictions, smooths action errors over 20 timesteps, and marks memory as useful when the no-memory error exceeds ten times the memory-policy error. Labels can activate before the visibly critical moment because the prediction target is an entire future action chunk.
8.8 Why Not Use Continuous Gating
Continuous gates trained jointly with the policy also tend to saturate or require task-sensitive regularization. A separately calibrated continuous gate can perform similarly, but fractional gates prevent the clean computational skip available with a binary decision.
8.9 Training Cost and Calibration Overhead
Calibration can use early checkpoints and only part of the validation data; full convergence of the two preliminary policies is unnecessary. The authors recommend testing an always-on memory policy first and adding calibration when overfitting is apparent. The preserved table reports the additional training cost.
| Train | Train | Rollout | Train Gate | Train GMP |
| 1.2h | 6.7h | 6.4h | 1.3h | 31.2h |