Literature
Display
← All papers

Diversity You Can Actually Measure: A Fast, Model-Free Diversity Metric for Robotics Datasets

Sreevardhan Sirigiri · Nathan Samuel de Lara · Christopher Agia · Florian Shkurti · Fabio Ramos

Preprint posted · CoRL 2026 (accepted) ↗

Robotics13 min text · CondensedOriginal paper ↗
Contents

At a glance

FAKTUAL measures demonstration diversity through a signature kernel over whole trajectories, then chooses a smaller training set using entropy and determinant objectives. It needs neither a target policy nor online rollouts for selection.

Across simulated manipulation and four physical tasks, selected subsets often improve on random selection and sometimes on all available data. Results depend on task, budget, representation, and demonstration quality. This edition covers the complete 37-page revised paper, including Appendices A–I.

1. Introduction

Demonstration count hides redundancy, uneven behavior coverage, and differences in local density. A useful diversity measure must compare variable-length sequences of states, actions, and visual observations without reducing everything to independent frames.

The paper adapts kernel-spectrum entropy, closely related to the Vendi score, to trajectory signatures. Diversity complements quality and sufficient nearby examples; it is not itself a success or quality score [1] [16] [17]. Figure 1 preserves the pipeline.

Complete original figure 1
Figure 1: A graphical depiction of FAKTUAL. For each demonstration we embed the observations (RGB images) into a ViT feature space, or alternatively extract low-dimensional representations of the objects of interest (e.g. point clouds). We then flatten any additional modalities (states, actions) if required, so each demonstration is represented as a path/trajectory in space (details in Appendix B). Using these path representations, we compute signature-based entropy/diversity metrics and select a subset of the dataset that maximizes entropy or diversity.

2. Signature transform and kernel

A path signature collects iterated integrals that encode sequential geometry. A signature kernel compares those representations without necessarily constructing their full, infinite tensors. Definitions 1 and 2 remain unchanged below.

Truncation, dynamic programming, random features, and a differential-equation formulation offer different computational trade-offs [23] [24] [25]. Reparameterization invariance can remove speed variation, but signatures also have translation and tree-like ambiguities; the appendix discussion needs those qualifications.

Complete original definition 1Complete original definition 2

3. Problem statement

The setting is behavior cloning from successful state–action–observation trajectories. The retained objective averages loss within each demonstration and then across demonstrations, so unequal trajectory lengths do not automatically determine example weights.

The goal is a dataset-only statistic associated with closed-loop policy success and useful for subset selection. Similar entropy does not establish equivalent policy performance; that connection is assessed empirically, not proved.

Equation
Complete original equation bc

4. Measuring diversity

Normalize the signature Gram matrix to unit diagonal, divide by the number of demonstrations, and compute entropy from its nonnegative eigenvalues. They sum to one. The spectral Shannon entropy equals the matrix von Neumann entropy; exponentiating gives an effective rank/Vendi score [1] [30].

A second objective uses the Gram determinant to reward linearly independent feature directions. Geometrically, a Gram determinant is squared volume; the source calls it volume. Its behavior depends on the chosen kernel and representation, and unit-diagonal normalization alone does not ensure every widely separated pair has near-zero similarity.

Complete original definition 3Complete original definition 4

5. Data curation

Given a budget m, approximate the entropy-maximizing subset with greedy-style selection. Observed entropy saturation suggests redundancy in some datasets; Figure 2 shows rapid saturation for Can and little saturation for Transport over the tested range.

An optional mixture first selects roughly a fraction p of the budget by entropy, then selects the remainder from unused demonstrations by determinant. The authors report useful p values between 0.5 and 0.9, while determinant-only selection can also work. Appendix F preserves the alternative solvers and their conditional guarantees.

Complete original figure 2
Figure 2: Entropy vs. number of demonstrations. For Can, entropy saturates quickly and approaches an asymptote. For Transport it does not saturate over the range shown, indicating Transport is more diverse (at least as discriminated by the signature entropy) and each demo contributes almost equally to entropy, yielding weaker separation between random and FAKTUAL.
Complete original definition 5

6. Experiments and results

Experiments vary the demonstration budget, compare selection methods, and relate entropy to policy success. The plots generally report each seed’s maximum success across 20 evaluated checkpoints, followed by averaging across three seeds. Those are selected-checkpoint results, not final-checkpoint performance or confidence intervals.

6.1 Simulated experiments

RoboMimic uses Can and Square with proficient-human and multi-human datasets, plus multi-human Transport. Can uses recurrent behavior cloning; Square and Transport use diffusion policies. MetaWorld fine-tunes SmolVLA on Door Open, Stick Push, and Shelf Place [16] [33] [38] [39].

Can shows a clearer advantage than the harder tasks. Table 6 reports Can PH at 120 demonstrations as 0.95 versus 0.86 for random, but Square PH at 60 demonstrations as 0.47 versus 0.55. Transport MH at 180 demonstrations is also lower, 0.14 versus 0.16.

Figure 4 reports Pearson correlations of 0.74–1.00 between entropy and success. The plotted subsets also vary in size, so this association does not isolate a causal effect of diversity at fixed budget. MetaWorld gains are mixed: Door Open is slightly worse than random at budgets 35 and 45, while Shelf Place remains low for every method.

Complete original figure 3
Figure 3: RoboMimic curation results. Success rates are computed over 50 rollouts for each of 20 checkpoints; we report the maximum across checkpoints, averaged over three random seeds, with error bars indicating min/max values. On Can, FAKTUAL consistently outperforms random even at the lowest demo counts, with a particularly large gap for PH where performance improves sharply with very few demonstrations. For Square, curves are largely indistinguishable at low counts; at higher counts MH intervals still slightly overlap while PH shows a small but consistent advantage for FAKTUAL. This PH advantage is plausibly because PH comprises proficient-human, high-quality demos and FAKTUAL is a diversity-based (not quality-based) strategy (consistent with the uniform coverage/density discussion in Section 1 when data quality is high). Transport is more challenging: at small budgets FAKTUAL is comparable to or marginally below random, but trends toward an advantage as demonstrations increase. FAKTUAL may eventually outperform random as more demonstrations are added and entropy begins to saturate (Figure 2).
Complete original figure 4
Figure 4: Signature entropy correlates with success. Success rate vs. signature entropy across RoboMimic tasks (Can, Square, Transport) and dataset variants (MH/PH). Points are curated subsets colored by demonstration count; dotted lines are linear fits for random (green circles) and FAKTUAL (magenta squares). Pearson r per panel indicates a strong positive association between entropy and downstream success.
Complete original figure 5
Figure 5: MetaWorld curation results. Success rates over 50 rollouts per checkpoint, max across 20 checkpoints. MetaWorld [33] reports substantially different success rates across seeds; we plot only the mean over 3 seeds, with full per-seed results in Table 5.

6.2 Real-world experiments

The SO-ARM101 experiments cover drawer opening, placing a marker in a holder, dragging a mug with a hook, and conditionally organizing a marker or charging brick. SmolVLA is used throughout. Figure 6 retains every panel and the complete original caption.

The Figure 6 caption claims improvements over all data on three tasks, but its printed bars show strict gains on two: organization (34.7% versus 28.7%) and marker placement (48.0% versus 42.0%). Drawer opening ties at 100%, and mug dragging ties at 37.3%. The original caption remains intact. These four laboratory tasks do not establish robustness to pathological data or broad deployment generalization. Appendix G records a separate checkpoint-schedule conflict.

Complete original figure 6
Figure 6. Complete original, including its original caption or continuation heading.

6.3 Baselines

Comparisons include DemInf, CUPID, Demo-SCORE, an oracle with privileged labels, Success Similarity, and random selection [19] [22] [21]. Table 1 distinguishes auxiliary models, policy dependence, and online rollouts.

FAKTUAL generally beats random but remains below several stronger quality- or rollout-informed baselines in Figure 7. Its appeal is lower selection overhead. “Model-free” here means no learned scoring model or target-policy dependence; visual representations may still use a pretrained ViT.

Figure 7 uses low-dimensional observations instead of the ViT representations used elsewhere. Its absolute success rates therefore should not be directly compared with the earlier image-based experiment. The paper distinguishes selecting a subset from filtering low-quality demonstrations.

Complete original figure 7
Figure 7: RoboMimic curation results with baselines. Success rates over 50 rollouts per checkpoint, max across 20 checkpoints, averaged over three seeds; error bars indicate min/max. Relative to random pruning, FAKTUAL is generally better and can reach (and in spots slightly exceed) the “All demos” reference, but remains below most stronger rollout/quality-informed baselines and sits closest to Demo-SCORE’s band. We expect quality-based curation to outperform diversity-based curation when the dataset mixes high- and low-quality demonstrations (e.g., MH). The key trade is practical: FAKTUAL is model-free and lightweight, providing a fast “good-enough” curation pass when policy/rollout-dependent scoring or an auxiliary model is too expensive.
Complete original table 1
Table 1: Data curation strategy characteristics

6.4 Observations

Diversity can hurt at small budgets when hard tasks need many nearby examples. The authors interpret the crossover as a balance between coverage and local density; they do not prove that entropy alone determines that balance.

Selection reduces the marker-location imbalance in one task. It fails to distinguish five drawer-lighting conditions and does not systematically prefer one object-organization goal. These observations show that the representation decides which forms of diversity are visible.

7. Ablations

Increasing signature depth can help or hurt: fine trajectory detail may carry useful information or noise. Figure 12 varies signature levels, with budget m=120 and p=0; the main text’s phrase “improves with size m” does not match that fixed-budget plot.

The source reports useful entropy/determinant mixing across p=0.5–0.9 and says a global-alignment-kernel alternative performs like random. It provides no comparable detailed plot for those two claims.

8. Conclusion and limitations

FAKTUAL offers a policy-independent way to select less redundant demonstrations. The evidence supports a useful heuristic on the tested datasets, with quality-aware methods often stronger and low-budget diversity sometimes harmful.

All demonstrations are successful; adversarial or highly unusual harmful trajectories were not tested. The entropy–performance association is not causal, and there is no theory explaining saturation, the optimal subset size, or post-saturation degradation. Combining quality, balanced coverage, and local density remains a proposal.

B. Demonstrations as paths

Appendix B describes three ways to represent demonstrations. Their choices of modalities, visual encoder, normalization, camera, and temporal sampling affect the measured diversity.

B.1 Flatten and stack

Embed each image with a fixed ViT or extract a low-dimensional observation, then concatenate it with states, actions, rewards, and other per-step features. The retained vector gives the exact construction. Full trajectories can have different lengths.

Equation
Complete original formula stacked

B.2 Combine modality kernels

Alternatively compute a separate signature kernel for each modality and combine them with nonnegative weights summing to one. This preserves positive semidefiniteness and allows modality-specific hyperparameters. The authors report no improvement over stacking in their experiments.

Equation
Complete original formula modalities

B.3 Initial observation and motion

A cheaper variant mixes an RBF kernel on the initial visual observation with a signature kernel on the end-effector path. It separates initial scene diversity from motion diversity, but can miss distractors or scene changes that occur later.

Equations
Complete original equation 1
Complete original formula mixture

B.4 Implementation practices

Temporal subsampling reduces work but can miss short-lived changes. The authors recommend one camera covering the whole scene and warn that wrist-camera motion can inflate apparent diversity. This curation recommendation is distinct from the multi-camera policy inputs used in the real tasks.

The original time-augmentation example is preserved below, but its conclusion is incorrect for the stated plain path signature: (t,a) and (t,b) differ by a constant translation and still have identical increments and signatures. Keeping starting-point information requires an additional construction, such as basepoint augmentation; see the cited primer [13]. Time augmentation alone addresses a different invariance.

Equation
Complete original formula time example

C.1 Entropy

Shannon entropy averages the negative log probability of outcomes. Here the probability vector is supplied by the normalized kernel spectrum rather than a fitted distribution over demonstrations. Equation 2 is retained.

Equation
Complete original equation 2

C.1.1 Von Neumann entropy

The matrix analogue is minus the trace of a unit-trace positive semidefinite matrix times its matrix logarithm. Its spectral form connects directly to Shannon entropy. Zero eigenvalues use the limiting convention 0 log 0=0.

Equation
Complete original equation 3

C.2 Submodularity

Definitions 6–8 retain set functions, monotonicity, and diminishing marginal returns. Later approximation ratios require these properties; empirical saturation by itself is not a proof that the normalized spectral-entropy objective is monotone and submodular.

Complete original definitions 6 8

D.1 Signature transform

Definition 9 retains the tensor algebra and iterated-integral construction for continuous paths with finite variation assumptions. The original mathematical statements remain available for inspection.

Complete original definition 9

D.1.1 Properties

Reversing a path gives its tensor inverse. Uniqueness must account for translation, time reparameterization, and tree-like equivalence; the source’s simple claim of injectivity for “non-tree-like paths” omits these qualifications [13].

Equation
Complete original formula reversal

Chen’s identity

Chen’s identity composes successive path segments through tensor multiplication. The retained theorem includes both the whole-signature and levelwise forms, followed by the group-like coproduct relation.

Complete original theorem 1
Equation
Complete original formula group like

Truncation

Finite truncation reduces the feature representation to finitely many tensor levels. The original decay bound and Equation 5 remain below. The stated general p-variation factorial bound should be checked against the cited rough-path source before using it as a numerical error guarantee; the paper does not derive it [29].

Equations
Complete original formula decay
Complete original equation 5

D.2 Signature kernel

Definition 10 takes an inner product between signatures. Theorem 2 expresses the untruncated kernel for differentiable paths as a second-order hyperbolic PDE with unit boundary values [23].

Pairwise kernel evaluation and forming an entire dataset Gram matrix are different costs. The latter still requires quadratically many pairs. The source’s effectively linear per-pair GPU wall time assumes sufficient parallel hardware and does not make the full dataset calculation linear.

Complete original definition 10Complete original theorem 2

E. Random Fourier signature features

Appendix E presents random Fourier signature features from Tóth and colleagues [25]. It fixes sequence increments, ordered index sets, bounded variation, and a static kernel, then gives the discrete truncated expansion in Equation 7.

Random features reduce the per-sequence representation cost. Forming and repeatedly processing a dense Gram matrix can still dominate large-dataset curation; the paper supplies no end-to-end timing table establishing negligible overhead at arbitrary scale.

Equations
Complete original formula rfsf notation
Complete original equation 7

E.1 Random Fourier features

A translation-invariant kernel can be approximated by sine/cosine features sampled from its spectral measure. Gaussian kernels use Gaussian spectral weights. The original Fourier map and moment assumption in Equation 8 remain unchanged.

Equations
Complete original formula bochner
Complete original formula rff map
Complete original equation 8

E.2 Signature features

Independent random-feature realizations across tensor levels avoid the bias of a naive shared-feature construction. Definition 11 gives the complete map and kernel; Theorem 3 gives concentration under compact-domain, variation, and moment assumptions.

The theorem concerns kernel approximation, not policy success or a guaranteed optimal curated set. Explicit tensor features grow rapidly with feature dimension and truncation depth, motivating the two projections below.

Complete original definition 11Complete original theorem 3

E.3 Diagonal projection

RFSF-DP restricts the outer feature-index sum to diagonal combinations. It remains unbiased while reducing feature dimension to linear dependence on random-feature width, with exponential dependence on signature depth. Definition 12, Theorem 4, and the stated computational cost are preserved.

Complete original definition 12Complete original theorem 4
Equations
Complete original equation 9
Complete original formula dp cost

E.4 Tensor random projection

RFSF-TRP projects tensor products to lower dimension and evaluates them through Hadamard products without materializing full tensors. Definition 13 and Theorem 5 retain the construction and its concentration bound; approximation to the true kernel additionally uses the earlier random-feature bound.

The reported cost is linear in sequence length but contains a quadratic term in feature width. It should not be described as strictly linear in every dimension.

Complete original definition 13Complete original theorem 5
Equation
Complete original formula trp cost

E.5 Computational trade-offs

Table 2 preserves feature dimensions, per-sequence costs, and concentration rates. The last paragraph names a DP empirical kernel but calls the ablation implementation TRP; that notation mismatch is retained and should be resolved against implementation before reproduction.

Entrywise kernel approximation does not by itself establish accuracy of entropy, determinant rankings, or downstream policy performance without further stability analysis.

Complete original table 2
Table 2: Computational/statistical trade-offs for the three RFSF variants of [25].

F. Subset algorithms

Appendix F formulates fixed-budget subset optimization and introduces a regularized log determinant for numerical stability. Its claim that adding μI leaves the optimizer unchanged is not generally justified: a log transform preserves determinant rankings, but diagonal regularization can change them.

Exact search is proposed for small datasets; greedy, stochastic, and determinantal sampling are alternatives for larger ones. The text invokes maximum coverage and integer-programming hardness without giving a reduction specific to each stated objective.

Equations
Complete original formula subset objective
Complete original formula logdet

F.1 Greedy local search

Algorithm 1 greedily adds the greatest marginal gain and optionally performs improving single-element swaps. It stops early when the best gain is nonpositive, so its printed code can return fewer than m elements.

The familiar 1−1/e guarantee is conditional on a nonnegative monotone submodular objective under a cardinality upper bound. It is not an unconditional guarantee for FAKTUAL’s normalized entropy objective [75] [76].

Complete original algorithm 1

F.2 Stochastic greedy

Algorithm 2 evaluates gains only within a fresh random candidate subset at each iteration. The stated expected approximation factor is 1−1/e−ε under the same objective assumptions; its reduced function-evaluation count does not include the cost of each entropy or determinant evaluation [77].

Complete original algorithm 2

F.3 Determinantal sampling

Algorithm 3 samples a size-m determinantal point process: choose eigenvectors using elementary symmetric polynomials, then sample items while updating an orthogonal basis. The complete 26-line algorithm is retained [78].

Sampling favors high-determinant subsets but is not maximum-a-posteriori optimization. With diagonal regularization, the determinant being sampled is that of the regularized kernel.

Complete original algorithm 3
Equation
Complete original formula dpp

G.1 RoboMimic setup

Can transfers a can between bins; Square places a nut over a rod; Transport requires two arms to transfer a hammer while clearing a target bin. Figure 8 retains all three scenes.

PH has 200 demonstrations from one experienced teleoperator. MH has 300 from six operators across three proficiency groups. The setup states train/validation splits of 180/20 and 270/30, whereas Table 6 labels MH full data as 280; that unresolved difference is preserved.

Complete original figure 8
Figure 8: Scene visualization of the 3 Robomimic tasks.

G.1.3 Policy settings

Table 3 is retained across both pages. Can uses a two-layer LSTM with 1,000 hidden units and a five-mode Gaussian mixture. Square and Transport use diffusion policies with 100 training and inference diffusion steps.

Batch size is 16. Training uses 100,000 steps for Can and Square and 200,000 for Transport; evaluation has 50 rollouts at each checkpoint. The main-text footnote’s claim of 100% Can full-data success differs from the Table 6 full-data values.

Complete original table 3 a
Table 3: Hyperparameters and model settings for the Robomimic tasks.
Complete original table 3 b
Table 3 b. Complete original, including its original caption or continuation heading.

G.2 MetaWorld setup

Door Open, Stick Push, and Shelf Place each use 50 demonstrations, with randomized task and goal positions. Figure 9 preserves all scenes. The paper says the cited sources do not disclose how those demonstrations were collected.

Complete original figure 9
Figure 9: Scene visualization of the 3 Metaworld tasks.

G.2.3 Policy settings

Table 4 retains the full SmolVLA configuration and exact backbone identifier. The vision encoder is frozen; the expert, small heads, and state projection are trained. Batch size is eight, training lasts 40,000 steps, and evaluation occurs every 2,000 steps with 50 rollouts.

Complete original table 4
Table 4: SmolVLA hyperparameters and training configuration for the Metworld tasks.

G.3 Real-world setup

Drawer and marker tasks contain 50 demonstrations each; object organization and mug dragging contain 100, from one experienced teleoperator. Marker evaluation adds two unseen locations to the five collection locations. Objects are never presented in both organization branches simultaneously.

Policies receive wrist and overhead views, plus a side view for drawer and organization tasks. Cameras record 640×480 RGB at 30 frames per second. Figures 10 and 11 preserve all task views and conditional starting scenes.

Complete original figure 10
Figure 10: Scene visualization of the drawer open task, pick and place marker task and mug drag task.
Complete original figure 11
Figure 11: Scene visualization of the organize objects tasks.

G.3.3 Policy settings

The real-task setup follows Table 4 with batch size 64, 60,000 training steps, and evaluation every 20,000 steps. That implies three scheduled evaluations, conflicting with Figure 6’s “20 checkpoints” caption. The source does not resolve which evaluation protocol produced that figure.

H.1 Signature-level results

Figure 12 retains both Can variants, three-seed min/max bars, m=120, and p=0. It varies signature depth rather than dataset size. The differing curve shapes support task-dependent truncation, not a universal best depth.

Complete original figure 12
Figure 12: Number of signature levels vs. success rate results. Results are averaged over three random seeds, and the error bars indicate the minimum and maximum values. Parameters (such as dataset size) are set to m = 120 and p = 0.

H.2 Full results

Tables 5 and 6 retain every seed, summary value, budget, full-data reference, and continuation page. Their min/max columns describe seed ranges, not statistical confidence intervals.

Two Stick Push FAKTUAL rows in Table 5 are inconsistent with their printed seeds: at 25 demonstrations the seeds 0.44/0.26/0.22 imply mean ≈0.307 and minimum 0.22, not 0.29/0.18; at 35 demonstrations 0.46/0.26/0.20 imply mean ≈0.307 and minimum 0.20, not 0.30/0.18. Original values are preserved rather than silently corrected.

Complete original table 5
Table 5: Success rates for all metaworld manipulation tasks over 3 seeds. Mean, minimum, and maximum are computed across seeds.
Complete original table 6 a
Table 6: Success rates for all robomimic manipulation tasks over 3 seeds. Mean, minimum, and maximum are computed across seeds.
Complete original table 6 b
Table 6 b. Complete original, including its original caption or continuation heading.

I. Proof

Lemma 1 and its complete proof show why normalized kernel-spectrum entropy equals matrix entropy. The trace is invariant under a change to the eigenvector basis.

The proof’s introductory sentence names eigenvalues of K, while subsequent equalities need eigenvalues of K/n. Read the λ values as the normalized spectrum, as specified in Definition 3; zero eigenvalues require the continuous 0 log 0 convention.

Complete original lemma 1Complete original proof 1
Concept demonstration

Open image ↗