Literature
Display
← All papers

DataMIL: Selecting Data for Robot Imitation Learning with Datamodels

Shivin Dass · Alaa Khaddaj · Logan Engstrom · Aleksander Madry · Andrew Ilyas · Roberto Martín-Martín

· ICLR 2026 ↗

Robotics10 min text · CondensedOriginal paper ↗
Contents

Overview

This edition covers the 25-page revised paper, including its estimator derivations, evaluation protocol, ablations, and compute costs. It preserves every numbered figure and table, both algorithms, all displayed formulas, and the original title mark.

DataMIL windmill mark from the original title

At a glance

DataMIL selects prior robot demonstrations by estimating how they affect a policy’s target-task performance. A datamodel assigns additive influence scores to data clusters, using held-out imitation loss as a differentiable substitute for costly environment rollouts. Policies trained with selected data improve average performance across simulation and physical manipulation experiments. The benefit is task dependent and comes with substantial offline computation.

1 Introduction

Large robot datasets can help specialization, but irrelevant or poor demonstrations can also hurt. Rather than equating usefulness with visual, language, or action similarity, DataMIL learns a model of how training-data inclusion changes a policy’s target objective. The robotics contribution is making that objective practical without online rollouts, reducing attribution noise through clustering, and addressing target-domain shift. The experiments study targeted data curation, not universal quality scores for every future robot task.

2 Related Work

Prior curation methods optimize broad dataset mixtures, score demonstration quality, or retrieve examples similar to a few target demonstrations. DataMIL instead adapts model-based data attribution to the target policy and task. It differs from rollout-based influence selection by evaluating its selection objective offline, although actual policy success is still measured with rollouts in the experiments. Related source: CUPID.

3 Preliminaries: Data Selection for Policies and Datamodels

The formal setup fixes the policy-learning procedure and target metric, then asks which subset of prior demonstrations produces the best trained policy. A cheap predictor of this training outcome replaces exhaustive retraining during subset search.

Refer to caption
Figure 1: Data selection with datamodels. (left) Similarity-based methods select close samples (yellow), but these aren’t always beneficial for learning. DataMIL evaluates data based on its impact on policy performance, selecting the samples that lead to policy improvement. (center) We estimate datamodels that score each sample by its influence on policy performance and select the highest-scoring samples for training. (right) DataMIL explores two datamodel estimation methods, adapted to robotics: regression and metagradient-based estimators (see Sec. 4.1).

Policy learning.

A learning algorithm maps a demonstration dataset to a stochastic state-to-action policy. A separate metric evaluates the resulting policy, typically through task success. Data quality is therefore relative to the learner, target task, and evaluation criterion.

Data selection in robotics.

The desired subset maximizes the target metric after training. Directly searching all subsets is infeasible because each candidate would require both policy optimization and evaluation.

Equation 1
arg​max𝒟′⊂𝒟⁡ℳ​(𝒜⁡(𝒟′)).\argmax_{\mathcal{D^{\prime}}\subset\mathcal{D}}\mathcal{M}(\mathcal{A}(\mathcal{D^{\prime}})). (1)

Datamodels.

A datamodel predicts policy performance from the subset’s membership vector. Linear datamodels assign additive coefficients to individual examples or clusters, making ranking and subset selection inexpensive once those coefficients are known. This is an approximation to training outcomes; the main difficulty is estimating useful coefficients, not solving the final ranking problem. Related source: Ilyas et al.’s datamodels.

Equation 2
arg​max𝒟′⊂𝒟⁡f^​(𝒟′).\argmax_{\mathcal{D^{\prime}}\subset\mathcal{D}}\hat{f}(\mathcal{D^{\prime}}). (2)

4 DataMIL: Datamodels for Robot Imitation Learning

The pipeline estimates a datamodel, adapts its objective and granularity to robotics, selects high-ranked prior data, and trains the downstream policy. Regression and metagradients provide two different accuracy–cost tradeoffs.

4.1 Estimating Datamodels

Regression trains policies on many random subsets and fits additive data coefficients to their observed scores. Metagradients instead differentiate a training-based target objective with respect to continuous data weights, giving a local first-order influence estimate. This avoids thousands of full retrainings for larger policies, but linear attribution remains approximate; the actual implementation differentiates only a final training segment, as specified in Appendix H. Related source: Engstrom et al.’s metagradient descent.

Equations 3–4
f^​(𝒟′)=∑zi∈𝒟′τ⁡(zi).\displaystyle\hat{f}(\mathcal{D}^{\prime})=\sum_{z_{i}\in\mathcal{D}^{\prime}}\tau(z_{i}).
{τ(z1),…,τ(zn)}:=argmin𝝉∈ℝn∑j=1m(∑j:zi∈Djτi−ℳ(𝒜(Dj)))2.\displaystyle\{\tau(z_{1}),\ldots,\tau(z_{n})\}:=\arg\min_{\bm{\tau}\in\mathbb{R}^{n}}\ \sum_{j=1}^{m}\left(\sum_{j:z_{i}\in D_{j}}\tau_{i}-\mathcal{M}(\mathcal{A}(D_{j}))\right)^{2}. (3)
ℳ⁡(𝒜⁡(𝐰))≈ℳ⁡(𝒜⁡(𝐰0))+∇𝐰ℳ​(𝒜⁡(𝐰0))⊤​(𝐰−𝐰0),\mathcal{M}(\mathcal{A}(\mathbf{w}))\approx\mathcal{M}(\mathcal{A}(\mathbf{w}_{0}))+\nabla_{\mathbf{w}}\mathcal{M}(\mathcal{A}(\mathbf{w}_{0}))^{\top}(\mathbf{w}-\mathbf{w}_{0}), (4)

4.2 Adapting Datamodels for Robotics

The proxy score is negative behavior-cloning loss on a small held-out target set, so larger scores mean better imitation. A MetaWorld comparison finds some loss of success relative to rollout-based attribution but large gains over using all data or target data alone. Temporal clustering reduces noisy per-example attribution: sub-trajectories suit LIBERO, whole trajectories the larger OXE collection. For physical tasks, half the target demonstrations enter datamodel training and half remain held out for scoring. The empirical usefulness of this proxy does not make imitation loss a guaranteed predictor of closed-loop success.

Equation 5
ℳ^(π,𝒟t​a​r​g​e​t)=1|𝒟t​a​r​g​e​t|∑(s,a)∈𝒟t​a​r​g​e​t−ℒB​C(π(s),a)\mathcal{\widehat{M}}(\pi,\mathcal{D}_{target})=\frac{1}{|\mathcal{D}_{target}|}\sum_{(s,a)\in\mathcal{D}_{target}}-\mathcal{L}_{BC}(\pi(s),a) (5)
Original figure
Figure 2: Comparing true rollout success (ℳ\mathcal{M}) vs. proxy metric (ℳ^\mathcal{\widehat{M}})

4.3 Data Selection and Policy Training

Rank clusters by estimated positive influence, retain a chosen fraction, then train using selected prior data and target demonstrations. The usual recipe samples the two sources with probabilities one minus alpha and alpha. MetaWorld is an explicit exception: its final policies use selected prior data alone because target co-training gave negligible improvement.

Original figure
Original figure
(a) Avg. success over 50 MetaWorld Tasks
Original figure
(b) Avg. success over LIBERO-10
Figure 3: Performance of policy trained on selected datasets in sim. environments.

5 Experiments

Evaluation spans 50 MetaWorld tasks, ten LIBERO target tasks, and four physical settings using OXE subsets. MetaWorld uses state-based Gaussian policies; LIBERO and OXE fine-tune pretrained Octo. Comparisons include state/action retrieval, behavior retrieval, flow retrieval, visual sub-trajectory retrieval, target-only training, and either all-data or budget-matched random selection. Related source: Octo.

5.1 Results

MetaWorld mixes expert demonstrations with lower-quality exploration data and provides five target demonstrations per task. LIBERO selects 10% of 4,500 prior demonstrations using metagradients; its mean success is 37.76%, versus 36.4% for behavior retrieval, with substantial task variation. Physical experiments cover ball placement, pouch opening, Tiago sink manipulation, and a three-task Droid setting. DataMIL has the highest reported mean across these settings, including transfer to Tiago despite its absence from the prior data. It does not win every task: STRAP is better on Franka-Ball, and several LIBERO tasks favor other methods.

Refer to caption
Figure 4: Results on OXE. We evaluate policies trained on subsets of the Open X-Embodiment dataset selected by different strategies. DataMIL consistently achieves the highest success, highlighting the need for end-end policy-aware data selection techniques. (Droid-Multitask shows average success; per-task results in App. C.7.)

5.2 What data is selected by DataMIL?

Selected-data distributions often span several source datasets rather than concentrating on one visually similar domain. For Tiago, useful data can come from different robots with related tabletop interaction structure; for the pouch task, frequently selected sources use Franka. High- and low-ranked images may look alike because their actions differ. These visual analyses suggest mechanisms but do not isolate causally why each example helps or harms.

Original PDF figure 5 with all panels, labels and source layout.
Figure 5: Distribution of datasets selected by different methods for Tiago-Sink task
Original PDF figure 6 with all panels, labels and source layout.
Figure 6: Distribution of datasets selected by different methods for Franka-Pouch task

6 Conclusion

The results support task-aware curation, with limits in computation, hyperparameter choice, and evaluation breadth. Metagradient estimation still costs several times ordinary policy training; selection fraction and cluster granularity need empirical tuning. Most targets are single tasks, with only a small multitask physical test, so generalist-scale curation remains unproven.

Acknowledgments

The authors acknowledge help with real-robot evaluations, manuscript feedback, and MetaWorld data generation. Funding includes DARPA’s TIAMAT program, the National Science Foundation, and Open Philanthropy.

Appendix A Qualitative Results: What data is selected by DataMIL?

The qualitative appendix examines which prior domains, tasks, and examples each method selects. It helps interpret the quantitative results without turning visual resemblance into a universal criterion of usefulness.

(a) Composition of data selected via DataMIL
Original figure
(b) Composition of data selected via SR
Original figure
(c) Composition of data selected via AR
Original figure
(d) Composition of data selected via BR
Original figure
Figure 7: MetaWorld Qualitative Results. Percentage of data selected from each task and expert/suboptimal source for DataMIL, SR, AR and BR

A.1 MetaWorld

For pick-place-wall, state retrieval identifies related tasks but admits poor actions, while action retrieval suppresses some noisy data but loses task identity. Combined state/action retrieval inherits both tendencies. DataMIL favors the relevant task’s expert data while avoiding much of the suboptimal pool in this example.

A.2 LIBERO

For book-caddy and bowl-cabinet, the prior collection lacks the exact target task. The authors manually identify related tasks and show that DataMIL often favors compatible object arrangements and scenes. These examples demonstrate related-task retrieval rather than exact target-demonstration recovery.

Refer to caption
Figure 8: LIBERO Book-Caddy. Proportion of prior tasks selected for the book-caddy task
Refer to caption
Figure 9: LIBERO Bowl-Cabinet. Proportion of prior tasks selected for the bowl-cabinet task

A.3 OXE

Highest- and lowest-ranked physical examples often have similar appearance. Their different action labels offer a plausible explanation for different influence scores, but the paper leaves a detailed account of these fine distinctions to future work.

Original PDF figure 10 with all panels, labels and source layout.
Figure 10: Top and bottom ranked samples by DataMIL for each of the real-world tasks

Appendix B Estimating Datamodels

The appendix formalizes subset prediction and presents the regression and metagradient estimators with their original pseudocode. These definitions separate the datamodel approximation from the policy-training procedure it predicts.

B.1 Formalizing Datamodeling

Binary vectors encode dataset inclusion, and the formal objective minimizes prediction error over randomly sampled subsets. An additive model is a dot product between the inclusion vector and learned coefficients. The formal half-inclusion distribution differs from the smaller inclusion probability used in the reported MetaWorld regression experiment.

Equations 6–7
𝔼𝐰∼Bernoulli​(12)N​[(ℳ⁡(𝒜⁡(𝐰))−f^​(𝐰))2],\mathbb{E}_{\mathbf{w}\sim\text{Bernoulli}(\frac{1}{2})^{N}}\left[\left(\mathcal{M}(\mathcal{A}(\mathbf{w}))-\hat{f}(\mathbf{w})\right)^{2}\right], (6)
f^​(𝐰)=𝐰⊤​β.\hat{f}(\mathbf{w})=\mathbf{w}^{\top}\mathbf{\beta}. (7)

B.2 Regression Estimator

Sample membership masks, train a policy on each selected subset, evaluate the target score, then regress those scores on the masks. This estimates joint data-attribution coefficients rather than scoring each demonstration in isolation. Accuracy improves with enough trained subsets, at the cost of repeating policy training many times.

Equation 8
minβ∈ℝN⁡1m​∑i=1m(ℳ⁡(𝒜⁡(𝐰i))−𝐰i⊤​β)2,\min_{\mathbf{\beta}\in\mathbb{R}^{N}}\frac{1}{m}\sum_{i=1}^{m}\left(\mathcal{M}(\mathcal{A}(\mathbf{w}_{i}))-\mathbf{w}_{i}^{\top}\mathbf{\beta}\right)^{2}, (8)
Algorithm 1 Regression Estimator
Original PDF algorithm 1 with all panels, labels and source layout.
Algorithm 2 Metagradient-based Estimator
Original PDF algorithm 2 with all panels, labels and source layout.

B.3 Metagradient-based Estimator

Continuous sample weights allow differentiation of a downstream metric through policy optimization. Metagradient descent repeatedly perturbs training weights, computes influence gradients, masks coordinate updates, and averages gradient information across iterations. Differentiability of both the learner and evaluation objective is required. The printed count update uses a descent sign while the main objective is a maximized score, and does not specify count projection or clipping; these source conventions are retained and need checking before implementation. Appendix H also limits differentiation to the final 100 training steps.

Equations 9–12
ℳ(𝒜(𝐰))≈ℳ(𝒜(𝐰0))+∇ℳ(𝒜(𝐰0))⊤(𝐰−𝐰0),\mathcal{M}(\mathcal{A}(\mathbf{w}))\approx\mathcal{M}(\mathcal{A}(\mathbf{w}_{0}))+\nabla\mathcal{M}(\mathcal{A}(\mathbf{w}_{0}))^{\top}(\mathbf{w}-\mathbf{w}_{0}), (9)
f^(𝐰)=ℳ(𝒜(𝐰0))+∇ℳ(𝒜(𝐰0))⊤(𝐰−𝐰0).\hat{f}(\mathbf{w})=\mathcal{M}(\mathcal{A}(\mathbf{w}_{0}))+\nabla\mathcal{M}(\mathcal{A}(\mathbf{w}_{0}))^{\top}(\mathbf{w}-\mathbf{w}_{0}). (10)
𝐠=∇𝐰ℳ​(𝒜𝐜′​(𝐰))|𝐰=𝟎,\mathbf{g}=\nabla_{\mathbf{w}}\,\mathcal{M}(\mathcal{A}^{\prime}_{\mathbf{c}}(\mathbf{w}))\Big|_{\mathbf{w}=\mathbf{0}}, (11)
𝐜←𝐜−sign⁡(𝐠)⊙𝐦,\mathbf{c}\leftarrow\mathbf{c}-\operatorname{sign}(\mathbf{g})\odot\mathbf{m}, (12)

Appendix C Comparisons and Additional Results

Additional experiments isolate the proxy objective, goal information, selected fraction, clustering, co-training mixture, and training duration. Numerical per-task results expose where average gains conceal weak or losing tasks.

Original figure
Figure 11: Extended Fig. 2 Results. Comparing true rollout success (ℳ\mathcal{M}) vs. proxy metric (ℳ^\mathcal{\widehat{M}})

C.1 Comparison: True Rollout success vs. Proxy Metric

Four MetaWorld tasks compare regression attribution from real rollout success with attribution from held-out imitation loss. The proxy loses some success but remains much better than the uncurated alternatives. The gap is visibly larger for dial-turn than for the other examples, so the proxy is useful rather than equivalent to rollout evaluation.

Original figure
Original figure
Figure 12: Avg. success on MetaWorld without goal conditioning.

C.2 MetaWorld No-Goal Conditioning Experiments

Masking simulator goal information reduces overall MetaWorld performance, while DataMIL retains the best mean among the compared selectors. Because the prior data already includes expert examples of the target tasks, this test asks whether retrieval can identify them without explicit goals; it is not generalization to wholly unseen tasks.

C.3 Comparison: Percentage of Data Selected

On a five-task LIBERO subset, selection is most useful at small budgets. DataMIL leads the baselines at 5% and 10%, while the gap narrows or reverses at 20–30%. The chosen 10% fraction is empirical, and larger selected datasets are not uniformly better.

Original PDF figure 13 with all panels, labels and source layout.
Figure 13: Ablation experiments of DataMIL over different key data selection and training hyperparameters. Each experiment is averaged over 5 seeds on each of the tasks in LIBERO-5. In (a) and (c) each data point represents an averaged success rate over all 5 tasks.

C.4 Comparison: Cluster Size

Fifteen-step sub-trajectories outperform thirty-step or full-trajectory grouping on the LIBERO-5 average. Finer clusters offer more selective control in this dataset; that result does not contradict using full trajectories for much larger OXE, where per-sample exposure is lower and attribution noisier.

C.5 Comparison: Co-Training Ratio

A balanced target/prior mixture, alpha equal to 0.5, gives the best LIBERO-5 results among the tested ratios. The same choice is transferred to hardware to avoid another expensive physical sweep; its optimality there is not separately established.

C.6 Comparison: Longer Training for All-Data Baseline

Doubling all-data training barely changes success, and quadrupling it reduces success in the reported five-task experiment. Final comparisons therefore hold training-step counts equal. This checks one potential explanation for the selection gain without proving that every alternative all-data training recipe would fail.

C.7 Numerical Results and Discussion

Table 1 reports full per-task outcomes. DataMIL’s highest mean does not imply task-wise dominance, and its mug-mug and mug-pudding results remain low. Comparisons with published STRAP numbers are affected by this study’s final-checkpoint-only evaluation and its use of Octo rather than STRAP’s original policy architecture. In the OXE rows, the all-data column represents the budget-matched random baseline described in the main text.

Tasks Target-Only All-Data AR BR Flow STRAP DataMIL (ours)
LIBERO Evaluations
Soup-Sauce 13.2±7.613.2\pm 7.6 20.0±15.620.0\pm 15.6 32.8±9.332.8\pm 9.3 38±15.338\pm 15.3 50±14.450\pm 14.4 33.2±15.733.2\pm 15.7 39.2±11.939.2\pm 11.9
Cream-Butter 27.6±6.527.6\pm 6.5 35.2±13.235.2\pm 13.2 39.6±10.739.6\pm 10.7 41.2±14.641.2\pm 14.6 47.2±10.847.2\pm 10.8 20.0±9.720.0\pm 9.7 50.4±8.650.4\pm 8.6
Stove-Moka 27.4±7.327.4\pm 7.3 24.0±9.824.0\pm 9.8 31.2±8.131.2\pm 8.1 30.4±10.030.4\pm 10.0 33.2±6.733.2\pm 6.7 43.6±5.543.6\pm 5.5 40.4±8.540.4\pm 8.5
Bowl-Cabinet 48.8±6.448.8\pm 6.4 65.6±8.365.6\pm 8.3 62.8±3.362.8\pm 3.3 73.6±8.573.6\pm 8.5 69.2±12.269.2\pm 12.2 77.2±10.377.2\pm 10.3 72.4±5.272.4\pm 5.2
Mug-Mug 0.4±0.90.4\pm 0.9 2.0±2.442.0\pm 2.44 4.8±2.34.8\pm 2.3 2.0±2.82.0\pm 2.8 6.4±6.56.4\pm 6.5 5.2±5.25.2\pm 5.2 0.8±1.10.8\pm 1.1
Book-Caddy 51.2±8.351.2\pm 8.3 58.4±10.758.4\pm 10.7 65.2±13.165.2\pm 13.1 76.8±7.676.8\pm 7.6 69.2±13.269.2\pm 13.2 83.2±7.883.2\pm 7.8 82.4±1.782.4\pm 1.7
Mug-Pudding 2.0±2.02.0\pm 2.0 2.4±1.72.4\pm 1.7 5.6±5.95.6\pm 5.9 8.4±2.28.4\pm 2.2 5.8±3.55.8\pm 3.5 10.8±6.910.8\pm 6.9 4.8±3.34.8\pm 3.3
Soup-Cheese 11.2±3.311.2\pm 3.3 22.0±6.522.0\pm 6.5 29.3±10.429.3\pm 10.4 35.2±5.235.2\pm 5.2 26.8±4.826.8\pm 4.8 35.2±3.935.2\pm 3.9 36.0±6.036.0\pm 6.0
Moka-Moka 5.6±4.35.6\pm 4.3 12±8.212\pm 8.2 4.8±2.34.8\pm 2.3 15.2±4.115.2\pm 4.1 7.6±5.57.6\pm 5.5 6.8±1.86.8\pm 1.8 12.0±5.512.0\pm 5.5
Mug-Microwave 27.6±13.527.6\pm 13.5 35.6±10.435.6\pm 10.4 29.2±10.629.2\pm 10.6 43.2±9.943.2\pm 9.9 38.8±5.238.8\pm 5.2 34.4±8.334.4\pm 8.3 39.2±12.739.2\pm 12.7
Libero-Average 21.521.5 27.7227.72 30.5230.52 36.436.4 35.4235.42 34.9634.96 37.76
OXE Evaluations
Franka-Ball 21.421.4 35.735.7 28.628.6 50.050.0 28.628.6 57.157.1 50.050.0
Franka-Pouch 17.617.6 17.617.6 11.811.8 41.241.2 35.335.3 29.429.4 70.670.6
Tiago-Sink 33.333.3 43.643.6 35.935.9 46.246.2 46.246.2 38.538.5 64.164.1
Droid (Drawer) 0.00.0 0.00.0 0.00.0 20.020.0 70.070.0 55.055.0 75.075.0
Droid (Bread) 4.24.2 33.333.3 20.820.8 0.00.0 41.741.7 16.716.7 41.741.7
Droid (Napkin) 25.025.0 45.045.0 40.040.0 45.045.0 40.040.0 35.035.0 65.065.0
Droid-Multitask 9.49.4 26.626.6 20.320.3 20.320.3 50.050.0 34.434.4 59.459.4
Real-Average 20.420.4 30.930.9 24.124.1 39.439.4 40.040.0 39.839.8 61.0
Table 1: Numerical results for LIBERO and OXE evaluations

Appendix D Proxy Metric Details

The proxy is the negative loss appropriate to the policy: action absolute error or Gaussian negative log-likelihood for MetaWorld, and diffusion denoising loss for Octo. Absolute error worked better with regression in the reported tests, while negative log-likelihood suited metagradients. These are offline supervised objectives evaluated on target demonstrations, not direct task-success measurements.

Equations 13–15
ℳ^(π,𝒟t​a​r​g​e​t)=1|𝒟t​a​r​g​e​t|∑(s,a)∈𝒟t​a​r​g​e​t−ℒB​C(π(s),a)\mathcal{\widehat{M}}(\pi,\mathcal{D}_{target})=\frac{1}{|\mathcal{D}_{target}|}\sum_{(s,a)\in\mathcal{D}_{target}}-\mathcal{L}_{BC}(\pi(s),a)
πθ​(a∣s)=𝒩⁡(a,μθ1​(s),diag⁡(σθ2​(s)2))\pi_{\theta}(a\mid s)=\mathcal{N}\!\bigl(a;\,\mu_{\theta_{1}}(s),\,\mathrm{diag}(\sigma_{\theta_{2}}(s)^{2})\bigr)
ℒNLL​(s,a)\displaystyle\mathcal{L}_{\mathrm{NLL}}(s,a) =−log⁡πθ​(a∣s)\displaystyle=-\log\pi_{\theta}(a\mid s)
=12​(a−μθ1​(s))⊤​Σ​(s)−1​(a−μθ1​(s))+12​log​det(2​π​Σ​(s))\displaystyle=\tfrac{1}{2}\,(a-\mu_{\theta_{1}}(s))^{\top}\Sigma(s)^{-1}\,(a-\mu_{\theta_{1}}(s))\;+\;\tfrac{1}{2}\,\log\det\bigl(2\pi\,\Sigma(s)\bigr) (13)
ℒℓ1(s,a)=∥a−μθ1(s)∥1\mathcal{L}_{\ell_{1}}(s,a)\;=\;\bigl\lVert a\;-\;\mu_{\theta_{1}}(s)\bigr\rVert_{1} (14)
ℒdiff(s,a)=𝔼t∼Uniform⁡[1,T],ϵ∼𝒩⁡(0,I)∥ϵ−ϵθ(α¯ta+1−α¯tϵ,s,t)∥2\mathcal{L}_{\mathrm{diff}}(s,a)=\mathbb{E}_{t\sim\mathrm{Uniform}[1,T],\,\epsilon\sim\mathcal{N}(0,I)}\Bigl\lVert\epsilon-\epsilon_{\theta}\bigl(\sqrt{\bar{\alpha}_{t}}\,a+\sqrt{1-\bar{\alpha}_{t}}\,\epsilon,\;s,\;t\bigr)\Bigr\rVert^{2}
ℳ^(π,𝒟target)=−1|𝒟target|∑(s,a)∈𝒟target𝔼t,ϵ∥ϵ−ϵθ(α¯ta+1−α¯tϵ,s,t)∥2\widehat{\mathcal{M}}(\pi,\mathcal{D}_{\mathrm{target}})\,=\;-\,\frac{1}{\lvert\mathcal{D}_{\mathrm{target}}\rvert}\sum_{(s,a)\in\mathcal{D}_{\mathrm{target}}}\mathbb{E}_{t,\epsilon}\Bigl\lVert\epsilon-\epsilon_{\theta}\bigl(\sqrt{\bar{\alpha}_{t}}\,a+\sqrt{1-\bar{\alpha}_{t}}\,\epsilon,\;s,\;t\bigr)\Bigr\rVert^{2} (15)

D.1 Up-weighting Relevant States

State-dependent weights allow task knowledge to modify the proxy. LIBERO doubles the weight of states immediately before grasping; other states keep unit weight. Thus the method can incorporate an explicit heuristic into its learned selection objective rather than being entirely free of manually chosen relevance assumptions.

Equation
ℳ^(π,𝒟t​a​r​g​e​t)=1|𝒟t​a​r​g​e​t|∑(s,a)∈𝒟t​a​r​g​e​t−w⁡(s,a)¯ℒB​C(π(s),a)\mathcal{\widehat{M}}(\pi,\mathcal{D}_{target})=\frac{1}{|\mathcal{D}_{target}|}\sum_{(s,a)\in\mathcal{D}_{target}}-\underline{w(s,a)}\mathcal{L}_{BC}(\pi(s),a)

Appendix E Baseline Implementation

Baselines use the authors’ retrieval implementations where possible, adapted to a common policy and selection budget. STRAP retrieves enough segments to cover 10% of LIBERO rather than its original fixed 100-segment recommendation; this improves its mean from 24.72% to 34.96% here. OXE input formatting is adapted without changing retrieval logic. MetaWorld baselines compare fifty-step temporal windows, which helped reject noisy examples. Related source: STRAP.

Appendix F Training and Evaluation Details

Training and evaluation differ across state-based simulation, visual simulation, and hardware. The following settings define what each reported average and success score actually measures.

F.1 MetaWorld

MetaWorld combines 350,000 expert transitions with one million subsampled exploration transitions from a multitask SAC agent. Five expert target demonstrations guide trajectory-level regression attribution; the top 10% of prior trajectories train the final Gaussian policy without target co-training. Results average three random seeds.

F.2 LIBERO

LIBERO-90 supplies 4,500 demonstrations; each of ten target tasks supplies five demonstrations. Influence is estimated for fifteen-step segments with the pre-grasp-weighted proxy. Octo-small is then fine-tuned for 10,000 steps with an equal target/selected mixture, evaluated on fifty rollouts per task, and averaged over five seeds.

F.3 OXE

OXE selection uses whole trajectories and task-specific subsets of 13, 23, or 24 source datasets. Target data is split for attribution, then reused in final equal-mixture Octo fine-tuning for 50,000 steps. Selection retains 0.5–1% of prior data. Physical evaluation fixes object configurations across methods and uses one seed, with 14 ball, 17 pouch, 39 sink, and 32 combined Droid trials. Droid grants half credit to partial completions, so its reported score is not strictly a full-task success fraction. Table 3 calls the ball task Franka-Pick. Related source: TeleMoMa.

Table 2: Datasets utilized across the OXE subsets in our experimental setup
Dataset OXE13 OXE23 OXE24 Dataset OXE13 OXE23 OXE24
RT1 ✓ ✓ ✓ CMU IAM Lab ✓ ✓ ✓
Viola ✓ ✓ ✓ Roboturk ✗ ✓ ✓
Austin Buds ✓ ✓ ✓ BC-Z ✗ ✓ ✓
Austin Mutex ✓ ✓ ✓ CMU Stretch ✗ ✓ ✓
Austin Sailor ✓ ✓ ✓ DLR Edan ✗ ✓ ✓
Austin Sirius ✓ ✓ ✓ Berkeley Autolab UR5 ✗ ✓ ✓
Taco Play ✓ ✓ ✓ Berkeley Fanuc ✗ ✓ ✓
Jaco Play ✓ ✓ ✓ Berkeley Cable ✗ ✓ ✓
Stanford Hydra ✓ ✓ ✓ Bridge ✗ ✓ ✓
NYU Franka ✓ ✓ ✓ NYU Door Opening ✗ ✓ ✓
Furniture Bench ✓ ✓ ✓ Toto ✗ ✓ ✓
UCSD Kitchen ✓ ✓ ✓ Kuka ✗ ✗ ✓
Table 3: Task-wise experimental setup for selecting data and, training and evaluating policies
Embodiment Prior Prior Selection No. of No. of
Dataset Ratio Target Demos Evaluations
Franka-Pick Franka-Panda OXE13 1% 10 14
Franka-Pouch Franka-Panda OXE23 0.75% 30 17
Tiago-Sink Tiago OXE24 0.5% 20 39
Droid-Multitask Franka-Panda OXE24 1% 40 (total) 32 (total)
    Drawer - - - 10 10
    Bread - - - 15 12
    Napkin - - - 15 10

Appendix G Computation Costs for Data Selection

Regression uses 10,000 trained subsets; the reported approximately eight hours across eight A100s corresponds to roughly 60–70 GPU-hours, amortized by reusing policies to score all fifty MetaWorld targets. Metagradients are typically three to five times slower than ordinary training. Reported selection costs include 16–20 hours on four A5000s for LIBERO and three to four days on four A100s for OXE. Avoiding robot rollouts shifts the cost to offline computation rather than eliminating it.

Appendix H Hyperparameters for estimating datamodels

MetaWorld regression samples inclusion masks at probability 0.1 and trains three-layer, 400-unit policies for 10,000 subset experiments. Metagradient experiments use 20 or 30 refinement steps, depending on the dataset. Crucially, differentiation covers only the final 100 policy-training steps, not the complete training history; exact differentiation of that segment should not be described as exact attribution through the full optimization.

Table 4: Hyperparameters for Metagradient Estimator.
Hyperparameter Metaworld LIBERO OXE
step size pp 1.0 1.0 0.125
number of steps TT 30 30 20
policy architecture gaussian MLP Octo-Small Octo-Small
policy learner 𝒜⁡(⋅)\mathcal{A}(\cdot)
   training steps 1100 2100 5100
   metagradient steps kk 100 100 100
   optimizer adamw adam adam
   lr schedular 1cycle cosine cosine
Concept demonstration

Open image ↗