Literature
Display
← All papers

When Does LeJEPA Learn a World Model?

David Klindt · Yann LeCun · Randall Balestriero

Preprint posted · NeurIPS 2026 (accepted) ↗

Learning theory12 min text · CondensedOriginal paper ↗
Contents

At a glance

Intuition. A map can preserve enough information for a classifier yet twist positions so that a straight line in the map becomes a curved path in the real world. This paper asks when a learned representation recovers the hidden state through only a rotation or reflection. Such a coordinate change preserves Euclidean distances and simplifies planning.

The guarantee concerns a specific world: independent Gaussian latent variables, stationary isotropic noisy transitions, matched latent/output dimension and an ideal population objective. The paper also studies approximate optimization and violations of these assumptions. Its Gaussian-uniqueness claim is within the stated world class, not a claim that other worlds cannot be learned by any method.

1 Introduction

The central question is identifiability: which hidden-state descriptions are determined by observations, positive-pair structure and the learning objective? Good downstream classification alone cannot determine this. The desired result is a learned coordinate system related linearly—and, here, orthogonally—to the true latent state.

Refer to caption
Figure 1: LeJEPA learns the World Model. (left) A world with independent Gaussian latent variables. (center) An unknown nonlinear process scrambles them into the data we observe. (right) LeJEPA [2] recovers the latent variables up to rotation. We prove this is the unique optimum.

Contributions.

The paper combines a forward recovery theorem, a converse within a specified transition family, an approximate-error bound and a coordinate-equivalence result for planning. Controlled experiments test each claim and separately explore natural images, extra predictors and incorrect output dimension.

2 Related Work

Representation Learning.

JEPA methods encourage related observations to agree while avoiding constant representations. This study compares different regularizers but focuses on which underlying state variables can be recovered, rather than treating probe accuracy as a complete world-model definition.

World Models.

World models span prediction, control and generative simulation. This result chiefly addresses the encoder’s state coordinates. Learning accurate action-conditioned dynamics and choosing a successful numerical planner remain additional problems.

Identifiability.

Identifiability needs structure in the data-generating process or learning problem. The study connects JEPA to slow-feature and independent-component analysis, while allowing broad measurable representation maps in its idealized analysis.

Complete original Figure 2 schematic from the exact-version PDF, retaining all panels, mathematical labels and vector artwork.
Figure 2: LeJEPA Theory Illustration. (left) The world has clean latent structure (Gaussian, disentangled) with correlated positive pairs. (center) An unknown nonlinear process produces the observations we actually see, scrambling the latent structure. (right) LeJEPA trains a representation with two objectives: pull positive pairs together (attract) and keep the embedding distribution Gaussian (SIGReg) to prevent collapse. We prove that the learned representation must be a rotation of the true latents, i.e., the representation is forced to learn the correct World Model (Thm. 5.1).

3 The World and the Learner

The world has hidden variables z, and an unknown observation process produces x. The encoder maps x to a learned state. Linear identifiability means the learned state retains the true state up to a linear coordinate change. It does not attach unique human names to individual coordinates.

3.1 The World

The setup specifies independent latent coordinates, stationary positive pairs and independent additive-noise transitions. These are modeling assumptions about how paired observations are generated. Ordinary adjacent robot frames need not satisfy them.

Assumptions (World)

(i)

Independence. p⁡(zi)⟂p⁡(zj)p(z_{i})\perp p(z_{j}) and transitions p⁡(zi′∣zi)⟂p⁡(zj′∣zj)p(z^{\prime}_{i}\mid z_{i})\perp p(z^{\prime}_{j}\mid z_{j}) for all i≠ji\neq j.

(ii)

Stationarity. Both views share the same marginal: p⁡(z)=p⁡(z′)p(z)=p(z^{\prime}).

(iii)

Additive noise. zi′=mi​(zi)+ηiz^{\prime}_{i}=m_{i}(z_{i})+\eta_{i} with ηi\eta_{i} independent of ziz_{i}.

3.1.1 The Gaussian World

The Gaussian case uses a common correlation parameter between zero and one for all coordinates: the next state is a contracted current state plus independent Gaussian noise of the matching variance. This preserves the standard Gaussian marginal. Equal transition scales are important to the spectral argument.

Equation 1
z′=ρ​z+1−ρ2​η,η∼𝒩⁡(0,IN),η⟂z,z^{\prime}=\rho\,z+\sqrt{1-\rho^{2}}\;\eta,\qquad\eta\sim\mathcal{N}(0,I_{N}),\quad\eta\perp z, (1)

3.2 The Learner: LeJEPA

The analyzed LeJEPA learner minimizes paired-state disagreement while keeping embedding covariance or distribution nondegenerate, with output dimension matching the latent dimension. Exact population optimality is an idealization. The theorem does not supply an optimization algorithm guaranteed to reach that optimum from finite pixels.

Equations 2–3
minh⁡ℒ⁡(h)=𝔼⁡[‖h⁡(z′)−h⁡(z)‖2]⏟Alignments.t.h⁡(z)∼𝒩⁡(0,IN)⏟Gaussianity\min_{h}\mathcal{L}(h)=\underbrace{\mathbb{E}\!\left[\|h(z^{\prime})-h(z)\|^{2}\right]}_{\textbf{Alignment}}\qquad\text{s.t.}\qquad\underbrace{h(z)\sim\mathcal{N}(0,I_{N})}_{\textbf{Gaussianity}} (2)
ℒ⁡(h)=2​N−2​∑i=1N𝔼⁡[hi​(z′)​hi​(z)].\mathcal{L}(h)=2N-2\sum_{i=1}^{N}\mathbb{E}\!\left[h_{i}(z^{\prime})\,h_{i}(z)\right]. (3)

4 Spectral Analysis of the World

The proof studies which functions of state stay correlated across a transition. These are analogous to frequency components, but adapted to the stochastic world.

Transition Operator.

The transition operator maps a function to its expected value at the next state, given the current state. Its eigenfunctions describe independently evolving patterns. Large eigenvalues correspond to patterns that remain more correlated across positive pairs.

Gaussian World: Hermite Polynomials.

In the Gaussian world, Hermite polynomials play the role of a natural basis. Linear patterns retain correlation ρ, quadratic ones ρ² and higher degrees still less. Since 0 < ρ < 1, mixing in nonlinear components reduces the best achievable alignment under the stated normalization.

Equation 4
𝔼⁡[hi​(z′)​hi​(z)]=w1⋅ρ+w2⋅ρ2+w3⋅ρ3+⋯≤ρ,\mathbb{E}\!\left[h_{i}(z^{\prime})\,h_{i}(z)\right]=w_{1}\cdot\rho+w_{2}\cdot\rho^{2}+w_{3}\cdot\rho^{3}+\cdots\;\leq\;\rho, (4)

General Case: Sturm-Liouville Theory.

For the more general transition family, a Sturm–Liouville equation describes the slow features. Requiring its relevant eigenfunction to be affine places a strong condition on the latent density, leading to the Gaussian converse.

5 Theory

Four results connect the encoder geometry to the world and then to planning. The manuscript supplies Lean verification files, with standard background results explicitly axiomatized; this is not a claim that every background theorem or software dependency was independently audited here.

5.1 Forward Direction: LeJEPA Learns the World Model

At the ideal optimum in the specified Gaussian world, the representation recovers the latent coordinates up to an orthogonal map. The original formal statement is retained.

Theorem 1 (LeJEPA Linear Identifiability)

Consider the Gaussian world (Sec. 3.1.1). Let h:ℝN→ℝNh:\mathbb{R}^{N}\to\mathbb{R}^{N} be any measurable map with h⁡(z)∼𝒩⁡(0,IN)h(z)\sim\mathcal{N}(0,I_{N}). Then ℒ​(h)≥2​(1−ρ)​N\mathcal{L}(h)\geq 2(1-\rho)\,N, with equality if and only if h⁡(z)=Q​zh(z)=Qz for some orthogonal Q∈O⁡(N)Q\in O(N). At any such optimum, h⁡(z′)|h⁡(z)∼𝒩⁡(ρ​h​(z),(1−ρ2)​IN)h(z^{\prime})\mid h(z)\sim\mathcal{N}(\rho\,h(z),(1-\rho^{2})\,I_{N}).

Proof Sketch.

The alignment objective becomes a sum of correlations. Each normalized coordinate reaches its maximum only when its Hermite expansion is linear. The covariance condition then forces the resulting linear matrix to be orthogonal.

Interpretation.

A global rotation or reflection preserves distances and the modeled transition. It may still mix what a person would call position, color or velocity; the result is not unique coordinate-by-coordinate semantic labeling.

5.2 Converse Direction: The Gaussian is Unique

The converse concerns the specified stationary additive-noise world class. The slow eigenfunction must be affine for the desired linear recovery, and that condition selects a Gaussian latent density under the stated assumptions.

Theorem 2 (Gaussian Uniqueness)

Consider any world satisfying Assumptions 3.1. Suppose every minimizer of (3) with Cov⁡(h⁡(z))=IN\mathrm{Cov}(h(z))=I_{N} is linear, h⁡(z)=Q​zh(z)=Qz. Then zz is Gaussian.

Proof Sketch.

Substituting an affine eigenfunction into the governing equation makes the derivative of log density linear. Integrating yields a quadratic log density, with normalizability selecting the Gaussian form.

5.3 Approximate Identifiability

An approximate result allows nonzero alignment error and covariance mismatch inside the same theoretical setting. It quantifies recovery error rather than assuming every trained checkpoint attains exact constraints.

Theorem 3 (Approximate Identifiability)

Consider the Gaussian world (3.1.1). Let h:ℝN→ℝNh:\mathbb{R}^{N}\to\mathbb{R}^{N} be measurable with 𝔼⁡[h⁡(z)]=0\mathbb{E}[h(z)]=0, satisfying: Approximate alignment: ℒ⁡(h)≤2​(1−ρ)​tr​(Cov⁡(h⁡(z)))+δ\mathcal{L}(h)\leq 2(1-\rho)\,\mathrm{tr}(\mathrm{Cov}(h(z)))+\delta; Approximate whitening: ‖Cov⁡(h⁡(z))−IN‖F≤ε\|\mathrm{Cov}(h(z))-I_{N}\|_{F}\leq\varepsilon. Define D=δ/(2​ρ​(1−ρ))D=\delta/(2\rho(1-\rho)). Then there exists Q∈O⁡(N)Q\in O(N) such that

Equation𝔼⁡[‖h⁡(z)−Q​z‖2]≤D+(ε+D)2\mathbb{E}\!\left[\|h(z)-Qz\|^{2}\right]\;\leq\;D\;+\;(\varepsilon+D)^{2}
(5)

Interpretation.

The bound separates nonlinear distortion from deviation of the linear part from orthogonality. Its dependence on the transition correlation matters; it is not a distribution-free guarantee for arbitrary non-Gaussian data.

5.4 Optimal Latent Planning

An orthogonal change of state coordinates preserves an equivalent control problem when dynamics and costs are transformed consistently. Rotation-invariant costs, such as Euclidean goal distance, are a convenient special case. This is a coordinate-equivalence theorem, not proof that a learned dynamics model or numerical optimizer is exact.

Theorem 4 (Optimal Latent Planning)

Let h⁡(z)=Q​zh(z)=Qz with Q∈O⁡(N)Q\in O(N), and let the costs be rotation-invariant:

Equationℓ⁡(R​z,a)=ℓ⁡(z,a),ℓT​(R​z)=ℓT​(z),∀R∈O⁡(N).\ell(Rz,a)=\ell(z,a),\qquad\ell_{T}(Rz)=\ell_{T}(z),\qquad\forall\,R\in O(N).
(6)

Then planning in the learned latent z^=h⁡(z)\hat{z}=h(z), with transitions pp mapped through hh and the same costs, yields the same value and plan: V^∗​(h⁡(z0))=V∗​(z0)\hat{V}^{*}\big(h(z_{0})\big)=V^{*}(z_{0}) and a^1:T∗(h(z0))=a1:T∗(z0)\hat{a}^{*}_{1:T}\big(h(z_{0})\big)=a^{*}_{1:T}(z_{0}).

Interpretation.

Equivalent true and learned-coordinate problems have matching action sequences and values. Additional learning or planning errors can still make a practical controller depart from that ideal.

6 Experiments

Experiments move from controlled latent mixtures to pixels, then deliberately violate dimension, distribution or predictor assumptions. Those latter results are empirical extensions rather than automatic consequences of the main theorem.

6.1 Forward: Linear Identifiability from Gaussian Latent Variables

Four invertible nonlinear mixing functions turn Gaussian latent states into observations. LeJEPA learns to undo those mixings up to rotation. One mixing preserves the Gaussian marginal itself, showing why Gaussianity alone is insufficient: the positive-pair alignment provides additional structure.

Original figure
Figure 3: 2D Simulations. Points colored by the polar angle and radius of the ground-truth latent variables z∼𝒩⁡(0,I2)z\sim\mathcal{N}(0,I_{2}) (like Fig. 1). Observations (a-c left) x=g⁡(z)x=g(z) after nonlinear mixing: parabolic shear, sinusoidal shear, RealNVP coupling layer [85]. Learned embeddings (a-c right): LeJEPA recovers the isotropic Gaussian structure up to rotation, consistent with Thm. 5.1.
Table 1: Scaling Comparison Across Regularizers (mean ±\pm std, 5 seeds). Three Gaussianity-enforcing objectives on a shared RealNVP mixing and matched encoder. SIGReg and VICReg maintain R2>0.999R^{2}>0.999 up to N=2048N{=}2048; InfoNCE matches at low NN but degrades at scale under fixed kernel width σ=1\sigma{=}1. At N=4096N{=}4096 all three fail at batch size 256; †raising the batch size to 1024 restores SIGReg (3 seeds). Per-method details in Tabs. 6–8.
Mixing Linear identifiability R2​(h→z)R^{2}(h\to z)
NN R2​(x→z)R^{2}(x\to z) SIGReg VICReg InfoNCE
±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−7\times 10^{-7} ±\pmstd ×10−7\times 10^{-7} ±\pmstd ×10−3\times 10^{-3}
2 0.781±\pm2.1 0.999998±\pm3.4 0.999996±\pm8.4 0.950961±\pm1.6
4 0.727±\pm24 0.999996±\pm12 0.999987±\pm54 0.910871±\pm8.2
8 0.728±\pm10 0.999993±\pm9.0 0.999988±\pm4.8 0.886818±\pm42
16 0.734±\pm6.3 0.999988±\pm4.9 0.999987±\pm4.6 0.999880±\pm0.01
32 0.737±\pm2.3 0.999981±\pm7.2 0.999981±\pm9.4 0.907809±\pm26
64 0.737±\pm1.5 0.999966±\pm7.4 0.999968±\pm8.1 0.648496±\pm3.1
128 0.739±\pm0.61 0.999938±\pm3.2 0.999942±\pm7.2 0.566955±\pm6.6
256 0.742±\pm0.49 0.999884±\pm7.9 0.999889±\pm7.2 0.696587±\pm0.49
512 0.749±\pm0.30 0.999775±\pm6.7 0.999785±\pm6.9 0.704393±\pm0.26
1024 0.763±\pm0.17 0.999561±\pm12 0.999582±\pm11 0.720241±\pm0.20
2048 0.768±\pm0.05 0.999043±\pm7.9 0.999094±\pm7.5 0.727140±\pm0.10
4096 0.768±\pm0.03 0.663 →\to 0.999449† 0.652 0.727111±\pm0.02

Scaling to High Dimensions.

Scaling experiments increase latent dimension through 4096 and compare SIGReg, VICReg and InfoNCE. Larger dimensions expose finite-batch and kernel-width limitations. Increasing the batch restores strong SIGReg recovery in the largest tested case, separating estimation issues from the population identifiability statement.

6.2 Converse: Non-Gaussian Latent Variables Break Linear Identifiability

The next tests change the latent distribution and sampling process to examine where linear recovery weakens.

Table 2: Linear Identifiability on RL Trajectories. (mean ±\pm std, 3 seeds). (left) Gaussian OU pairs recover true latents with rising R2R^{2}. (right) RL-policy trajectories [9] (δ\delta: frame separation) show reduced identifiability because of anisotropy (ρ0≠ρ1\rho_{0}\neq\rho_{1}) and non-Gaussian transitions (Figs. 14,15).
OU (Gaussian) Trajectory (non-Gaussian)
ρ\rho R2​(z→h)R^{2}(z\to h) R2​(h→z)R^{2}(h\to z) δ\delta ρ0\rho_{0} ρ1\rho_{1} R2​(z→h)R^{2}(z\to h) R2​(h→z0)R^{2}(h\to z_{0}) R2​(h→z1)R^{2}(h\to z_{1})
0.30 0.67±\pm2e-02 0.67±\pm2e-02 1 1.000 0.999 -0.39±\pm1e-01 0.71±\pm3e-02 -0.03±\pm4e-02
0.50 0.86±\pm1e-02 0.86±\pm1e-02 2 0.999 0.996 -0.47±\pm5e-02 0.73±\pm4e-03 0.01±\pm2e-03
0.70 0.93±\pm7e-03 0.93±\pm7e-03 4 0.997 0.991 -0.05±\pm3e-01 0.51±\pm1e-01 0.43±\pm9e-02
0.80 0.94±\pm3e-04 0.94±\pm3e-04 8 0.992 0.982 0.50±\pm2e-02 0.80±\pm2e-03 0.78±\pm6e-03
0.90 0.95±\pm7e-04 0.95±\pm7e-04 16 0.981 0.963 0.44±\pm4e-02 0.63±\pm2e-02 0.87±\pm2e-02
0.95 0.95±\pm2e-04 0.95±\pm2e-04 32 0.959 0.928 0.45±\pm6e-02 0.62±\pm2e-02 0.81±\pm1e-02
0.99 0.95±\pm4e-04 0.95±\pm4e-04 64 0.915 0.863 0.44±\pm4e-02 0.55±\pm3e-02 0.77±\pm2e-02

Latent-Distribution Sweep.

A generalized-normal sweep varies the latent shape from heavy-tailed through Gaussian toward uniform. Recovery is strongest near the Gaussian setting under the tested transitions and objectives.

Pixel-Based RL Trajectories.

The Reacher experiment uses the same image renderer with either controlled Gaussian transitions or recorded policy trajectories. The latter violate several assumptions at once, including marginal shape, isotropic correlation and joint-limit behavior. Their weaker recovery cannot isolate non-Gaussianity as the only causal change.

Original figure
Figure 4: Experimental Results. (a) Bound Verification. SIGReg runs across grid, 2D, scaling, and gennorm α=2\alpha{=}2 lie below the diagonal, confirming Thm. 5.3. Two near-zero outliers reflect finite-sample noise. (b) Gaussian Optimality. Linear recovery, R2​(h→zCLOSER^{2}(h\to z), peaks at Gaussian, illustrating Thm. 5.2. SIGReg’s Gaussianization of hh is more robust to non-Gaussian latent variable distributions than whitening. (c) Control cost over K=30K=30 random start-goal pairs (path length ≥1\geq 1, ideal 11). The Gaussian encoder is statistically indistinguishable from the oracle; the Trajectory encoder is biased upward. (d) Control cost decreases with linear identifiability R2R^{2}, supporting Thm. 5.4.

6.3 Approximation Bound and Loss Predict Identifiability

Measured alignment gap and covariance mismatch are compared with recovery error under controlled runs. Loss can be a useful proxy in these settings, but estimating the bound itself needs the theoretical quantities and finite-sample care.

6.4 Linear Identifiability Enables Latent-Space Planning

Straight-line plans in a faithfully recovered latent space correspond closely to straight paths in the true joint space. Distorted representations produce curved or more costly paths. The experiment evaluates this particular goal-reaching geometry, not every robotic planning task.

Original figure
Figure 5: Linear Identifiability Enables Latent-Space Planning. Interpolation in each encoder’s latent space between fixed start and goal frames, decoded by nearest-neighbor retrieval (Fig. 19). top) Oracle (joint-space straight line) [84]. middle) Gaussian encoder (Sec. 3.1.1: OU, ρ=0.99\rho=0.99) tracks the oracle (overlaid) closely. bottom) RL trajectory [9] encoder (stride δ=8\delta=8) deviates.

6.5 Beyond the Assumptions: Latent Dimension, Natural Images, Predictors

Original figure
Figure 6: Misspecified latent dimension (true N=8N{=}8, 5 seeds). Left: recovery tracks M/NM/N below NN and saturates above. Right: for M<NM<N, MM latents are recovered well and the rest poorly.

Unknown latent dimension.

Too few output coordinates retain only a subspace in the tested setting; larger outputs eventually saturate recovery. Covariance participation ratio is explored as a dimension diagnostic. Its behavior here does not make intrinsic dimension universally identifiable from arbitrary embeddings.

Table 3: ImageNet-X Linear Probes. Mean balanced accuracy over the nine factors with >1000>1000 balanced samples (5 repetitions; full table in App. H.16).
LeJEPA DINO iBOT v2 v3 init pixels
data IN-1k 142M 1.7B – –
acc. .677 .589 .628 .656 .645 .545 .533

Natural images.

ImageNet-X factors are linearly probed in pretrained visual encoders. These empirical comparisons test observable factor information without access to the true generative state, so they do not verify all Gaussian-world assumptions on natural images.

Predictors.

Adding a learned linear or multilayer predictor preserves recovery in the tested controlled experiments. A sufficiently flexible predictor could absorb an encoder distortion, so the general nonlinear-predictor extension remains empirical here.

7 Limitations

Are the latents Gaussian?

Real latent factors need not be independent, Gaussian or Euclidean. Maximum-entropy and aggregation arguments motivate a prior but do not establish those properties from observations. Extending the theory to other worlds remains open.

Finite samples and optimization.

The results concern a population optimum, not sample-complexity or training-convergence guarantees. The approximation theorem controls stated errors inside its setup; finite-sample estimates can introduce apparent bound violations.

8 Discussion

Outlook.

Actions, interventions and learned dynamics introduce further questions beyond encoder recovery. Inverse dynamics may help retain controllable factors in settings not covered by the passive Gaussian argument.

Practical Implications.

The experiments suggest that data collection and objective estimation both affect recovery. Goal-directed trajectories and isotropic exploration can expose different parts of the same physical state space. No one regularizer is established as optimal across every data regime.

A Theoretical Foundation for World Models.

The useful contribution is a conditional link between representation geometry and equivalent control coordinates. Keep the world assumptions, encoder guarantee and additional dynamics-learning problem separate when applying it.

Acknowledgments

The authors acknowledge feedback, computing support and funding.

Appendix Overview

The appendices contain notation, proofs, connections to slow features, Lean verification scope and detailed experimental settings.

Contents Page
Appendix A Proof of Thm. 5.1 (linear identifiability), incl. Hermite background A
Appendix B Proof of Thm. 5.2 (Gaussian uniqueness) B
Appendix C Proof of Thm. 5.3 (approximate identifiability) C
Appendix D Proof and discussion of Thm. 5.4 (optimal latent planning) D
Appendix E Alternative proof via Dirichlet energy E
Appendix F Prior work: connection to Slow Feature Analysis F
Appendix G Lean 4 formal verification G
Appendix H Experimental details and additional results H
Symbol Description
World Model
z,z′∈ℝNz,z^{\prime}\in\mathbb{R}^{N} True latent variables and positive pair (second view)
ziz_{i} ii-th coordinate of zz, one scalar latent variable
NN Latent dimension
MM Encoder output dimension (M=NM=N in theorems; App. H.13)
mim_{i} Deterministic part of the additive-noise transition, zi′=mi​(zi)+ηiz^{\prime}_{i}=m_{i}(z_{i})+\eta_{i}
ρ∈(0,1)\rho\in(0,1) Autocorrelation of the Ornstein–Uhlenbeck transition, mi​(zi)=ρ​zim_{i}(z_{i})=\rho\,z_{i}
η∼𝒩⁡(0,IN)\eta\sim\mathcal{N}(0,I_{N}) Independent transition noise
K>0K>0 Diffusion coefficient (Sturm–Liouville setting)
Maps
gg Unknown nonlinear mixing (generative) function; x=g⁡(z)x=g(z)
ff Learned encoder; y=f⁡(x)y=f(x)
h=f∘gh=f\circ g Composed map from latent space to representation space
Jh​(z)J_{h}(z) Jacobian matrix of hh at zz
Linear algebra
O⁡(N)O(N) Orthogonal group (matrices with Q​Q⊤=INQQ^{\top}=I_{N})
Q∈O⁡(N)Q\in O(N) Orthogonal recovery matrix (exact identifiability)
Losses and objectives
ℒ⁡(h)\mathcal{L}(h) Alignment loss: 𝔼⁡[‖h⁡(z′)−h⁡(z)‖2]\mathbb{E}[\|h(z^{\prime})-h(z)\|^{2}]
ℒinv\mathcal{L}_{\mathrm{inv}} Empirical invariance (alignment) loss
ℒSIG\mathcal{L}_{\mathrm{SIG}} SIGReg Gaussianity regularizer
λ\lambda Regularization weight balancing ℒSIG\mathcal{L}_{\mathrm{SIG}} and ℒinv\mathcal{L}_{\mathrm{inv}}
Hermite polynomials and spectral decomposition
Hek​(x)\mathrm{He}_{k}(x) Probabilist’s Hermite polynomial of degree kk
h^k​(x)\hat{h}_{k}(x) Normalized Hermite polynomial: Hek​(x)/k!\mathrm{He}_{k}(x)/\sqrt{k!}
α∈ℕN\alpha\in\mathbb{N}^{N} Multi-index (α1,…,αN)(\alpha_{1},\ldots,\alpha_{N}); |α|=α1+⋯+αN|\alpha|=\alpha_{1}+\cdots+\alpha_{N} is the total degree
Hα​(z)H_{\alpha}(z) Multivariate Hermite basis function: ∏jh^αj​(zj)\prod_{j}\hat{h}_{\alpha_{j}}(z_{j})
cα(i)c_{\alpha}^{(i)} Hermite coefficient of hih_{i} at multi-index α\alpha
wd(i)w_{d}^{(i)} Variance fraction of hih_{i} at degree dd: ∑|α|=d(cα(i))2\sum_{|\alpha|=d}(c_{\alpha}^{(i)})^{2}
TT Transition operator: (T​φ)​(z)=𝔼⁡[φ⁡(z′)∣z](T\varphi)(z)=\mathbb{E}[\varphi(z^{\prime})\mid z]
Approximate identifiability
δ≥0\delta\geq 0 Alignment gap (excess loss above the linear optimum)
ε≥0\varepsilon\geq 0 Covariance deviation: ‖Cov⁡(h⁡(z))−IN‖F\|\mathrm{Cov}(h(z))-I_{N}\|_{F}
DD Normalized alignment gap: δ/(2​ρ​(1−ρ))\delta/(2\rho(1-\rho))

Appendix A Proof of Theorem 5.1 (Forward Direction)

Theorem 5.1 (LeJEPA Linear Identifiability)

Consider the Gaussian world (Sec. 3.1.1). Let h:ℝN→ℝNh:\mathbb{R}^{N}\to\mathbb{R}^{N} be any measurable map with h⁡(z)∼𝒩⁡(0,IN)h(z)\sim\mathcal{N}(0,I_{N}). Then ℒ​(h)≥2​(1−ρ)​N\mathcal{L}(h)\geq 2(1-\rho)\,N, with equality if and only if h⁡(z)=Q​zh(z)=Qz for some orthogonal Q∈O⁡(N)Q\in O(N). At any such optimum, h⁡(z′)|h⁡(z)∼𝒩⁡(ρ​h​(z),(1−ρ2)​IN)h(z^{\prime})\mid h(z)\sim\mathcal{N}(\rho\,h(z),(1-\rho^{2})\,I_{N}).

A.1 Intuition: Why Should This Force Linearity?

An informal information argument motivates linearity but has gaps when transformed joint distributions are not Gaussian. The Hermite proof supplies the rigorous spectral route instead.

A.2 Hermite Polynomial Background

Definitions and orthogonality.

Hermite polynomials form an orthogonal basis for square-integrable functions under a Gaussian measure.

Equations 7–9
Hek(x)=(−1)kex2/2dkd​xke−x2/2,k=0,1,2,…\mathrm{He}_{k}(x)=(-1)^{k}\,e^{x^{2}/2}\,\frac{d^{k}}{dx^{k}}\,e^{-x^{2}/2},\qquad k=0,1,2,\ldots (7)
He0​(x)=1,He1​(x)=x,He2​(x)=x2−1,He3​(x)=x3−3​x\mathrm{He}_{0}(x)=1,\quad\mathrm{He}_{1}(x)=x,\quad\mathrm{He}_{2}(x)=x^{2}-1,\quad\mathrm{He}_{3}(x)=x^{3}-3x (8)
𝔼x∼γ​[Hej​(x)​Hek​(x)]=k!​δj​k.\mathbb{E}_{x\sim\gamma}\!\left[\mathrm{He}_{j}(x)\,\mathrm{He}_{k}(x)\right]=k!\,\delta_{jk}. (9)

Multivariate extension.

Products of one-dimensional Hermite polynomials extend the basis to independent Gaussian coordinates, grouped by total degree.

Equation 10
Hα​(z)=∏j=1Nh^αj​(zj).H_{\alpha}(z)=\prod_{j=1}^{N}\hat{h}_{\alpha_{j}}(z_{j}). (10)

Hermite expansion of measure-preserving maps.

Expansion coefficients divide each learned coordinate’s variance between linear and higher-degree components.

Equations 11–12
hi​(z)=∑|α|≥1cα(i)​Hα​(z),∑|α|≥1(cα(i))2=1.h_{i}(z)=\sum_{|\alpha|\geq 1}c_{\alpha}^{(i)}\,H_{\alpha}(z),\qquad\sum_{|\alpha|\geq 1}(c_{\alpha}^{(i)})^{2}=1. (11)
wd(i)=∑|α|=d(cα(i))2,∑d≥1wd(i)=1.w_{d}^{(i)}=\sum_{|\alpha|=d}(c_{\alpha}^{(i)})^{2},\qquad\sum_{d\geq 1}w_{d}^{(i)}=1. (12)

A.3 Proof

Setup and notation.

The proof rewrites normalized alignment as correlation and organizes each representation component by Hermite degree.

Equation 13
hi​(z)=∑|α|≥1cα(i)​Hα​(z),∑|α|≥1(cα(i))2=1.h_{i}(z)=\sum_{|\alpha|\geq 1}c_{\alpha}^{(i)}\,H_{\alpha}(z),\qquad\sum_{|\alpha|\geq 1}(c_{\alpha}^{(i)})^{2}=1. (13)

Step 1: The transition attenuates each Hermite degree by ρ d .

The Gaussian transition attenuates a degree d component by ρ to the power d, making higher degrees less persistent.

Equations 14–21
∑k=0∞tkk!​Hek​(x)=exp⁡(t​x−t22).\sum_{k=0}^{\infty}\frac{t^{k}}{k!}\,\mathrm{He}_{k}(x)=\exp\!\left(tx-\frac{t^{2}}{2}\right). (14)
∑k=0∞tkk!​Hek​(z′)=exp⁡(t⁡(ρ​z+1−ρ2​η)−t22).\sum_{k=0}^{\infty}\frac{t^{k}}{k!}\,\mathrm{He}_{k}(z^{\prime})=\exp\!\left(t(\rho z+\sqrt{1-\rho^{2}}\,\eta)-\frac{t^{2}}{2}\right). (15)
exp⁡(t​ρ​z−t22)⋅exp⁡(t​1−ρ2​η).\exp\!\left(t\rho z-\frac{t^{2}}{2}\right)\cdot\exp\!\left(t\sqrt{1-\rho^{2}}\,\eta\right). (16)
𝔼η​[exp⁡(t​1−ρ2​η)]=exp⁡(t2​(1−ρ2)2).\mathbb{E}_{\eta}\!\left[\exp\!\left(t\sqrt{1-\rho^{2}}\,\eta\right)\right]=\exp\!\left(\frac{t^{2}(1-\rho^{2})}{2}\right). (17)
t​ρ​z−t22+t2​(1−ρ2)2=t​ρ​z−t22+t22−t2​ρ22=t​ρ​z−(t​ρ)22.t\rho z-\frac{t^{2}}{2}+\frac{t^{2}(1-\rho^{2})}{2}=t\rho z-\frac{t^{2}}{2}+\frac{t^{2}}{2}-\frac{t^{2}\rho^{2}}{2}=t\rho z-\frac{(t\rho)^{2}}{2}. (18)
∑k=0∞tkk!​𝔼η​[Hek​(z′)]=exp⁡((ρ​t)​z−(ρ​t)22).\sum_{k=0}^{\infty}\frac{t^{k}}{k!}\,\mathbb{E}_{\eta}\!\left[\mathrm{He}_{k}(z^{\prime})\right]=\exp\!\left((\rho t)z-\frac{(\rho t)^{2}}{2}\right). (19)
exp⁡((ρ​t)​z−(ρ​t)22)=∑k=0∞(ρ​t)kk!​Hek​(z)=∑k=0∞tkk!​ρk​Hek​(z).\exp\!\left((\rho t)z-\frac{(\rho t)^{2}}{2}\right)=\sum_{k=0}^{\infty}\frac{(\rho t)^{k}}{k!}\,\mathrm{He}_{k}(z)=\sum_{k=0}^{\infty}\frac{t^{k}}{k!}\,\rho^{k}\,\mathrm{He}_{k}(z). (20)
𝔼η​[Hek​(z′)]=ρk​Hek​(z).\mathbb{E}_{\eta}\!\left[\mathrm{He}_{k}(z^{\prime})\right]=\rho^{k}\,\mathrm{He}_{k}(z). (21)

Step 2: Different Hermite components do not interact across views.

Orthogonality eliminates cross-terms between distinct basis components across the two views.

Equations 22–24
𝔼⁡[h^k​(z′)​h^j​(z)]=𝔼z​[𝔼η​[h^k​(z′)]​h^j​(z)].\mathbb{E}\!\left[\hat{h}_{k}(z^{\prime})\,\hat{h}_{j}(z)\right]=\mathbb{E}_{z}\!\left[\mathbb{E}_{\eta}\!\left[\hat{h}_{k}(z^{\prime})\right]\,\hat{h}_{j}(z)\right]. (22)
𝔼⁡[h^k​(z′)​h^j​(z)]=ρk​𝔼z​[h^k​(z)​h^j​(z)]=ρk​δj​k,\mathbb{E}\!\left[\hat{h}_{k}(z^{\prime})\,\hat{h}_{j}(z)\right]=\rho^{k}\,\mathbb{E}_{z}\!\left[\hat{h}_{k}(z)\,\hat{h}_{j}(z)\right]=\rho^{k}\,\delta_{jk}, (23)
𝔼⁡[Hα​(z′)​Hβ​(z)]=∏j=1N𝔼⁡[h^αj​(zj′)​h^βj​(zj)]=∏j=1Nραj​δαj​βj=ρ|α|​δα​β.\mathbb{E}\!\left[H_{\alpha}(z^{\prime})\,H_{\beta}(z)\right]=\prod_{j=1}^{N}\mathbb{E}\!\left[\hat{h}_{\alpha_{j}}(z_{j}^{\prime})\,\hat{h}_{\beta_{j}}(z_{j})\right]=\prod_{j=1}^{N}\rho^{\alpha_{j}}\,\delta_{\alpha_{j}\beta_{j}}=\rho^{|\alpha|}\,\delta_{\alpha\beta}. (24)

Step 3: Nonlinearity strictly reduces correlation.

Any nonzero higher-degree variance therefore reduces the attainable correlation relative to a linear component.

Equations 25–28
𝔼⁡[hi​(z′)​hi​(z)]=∑|α|≥1∑|β|≥1cα(i)​cβ(i)​𝔼​[Hα​(z′)​Hβ​(z)]=∑|α|≥1(cα(i))2​ρ|α|.\mathbb{E}\!\left[h_{i}(z^{\prime})\,h_{i}(z)\right]=\sum_{|\alpha|\geq 1}\sum_{|\beta|\geq 1}c_{\alpha}^{(i)}c_{\beta}^{(i)}\,\mathbb{E}\!\left[H_{\alpha}(z^{\prime})\,H_{\beta}(z)\right]=\sum_{|\alpha|\geq 1}(c_{\alpha}^{(i)})^{2}\,\rho^{|\alpha|}. (25)
𝔼⁡[hi​(z′)​hi​(z)]=w1⋅ρ+w2⋅ρ2+w3⋅ρ3+⋯\mathbb{E}\!\left[h_{i}(z^{\prime})\,h_{i}(z)\right]=w_{1}\cdot\rho+w_{2}\cdot\rho^{2}+w_{3}\cdot\rho^{3}+\cdots (26)
w1⋅ρ+w2⋅ρ2+w3⋅ρ3+⋯≤w1⋅ρ+w2⋅ρ+w3⋅ρ+⋯=ρ⋅(w1+w2+⋯)=ρ.w_{1}\cdot\rho+w_{2}\cdot\rho^{2}+w_{3}\cdot\rho^{3}+\cdots\leq w_{1}\cdot\rho+w_{2}\cdot\rho+w_{3}\cdot\rho+\cdots=\rho\cdot(w_{1}+w_{2}+\cdots)=\rho. (27)
hi​(z)=∑j=1Ncej(i)​zj=⟨qi,z⟩h_{i}(z)=\sum_{j=1}^{N}c_{e_{j}}^{(i)}\,z_{j}=\langle q_{i},z\rangle (28)

Step 4: Assembly.

At equality every coordinate is linear, and the covariance constraint makes the combined matrix orthogonal.

Equation 29
∑i=1N𝔼⁡[hi​(z′)​hi​(z)]≤ρ​N,\sum_{i=1}^{N}\mathbb{E}[h_{i}(z^{\prime})h_{i}(z)]\leq\rho N, (29)

Step 5: Recovery of the transition.

Substitution shows that an orthogonal coordinate change preserves the specified Gaussian transition law.

Appendix B Proof of Theorem 5.2 (Gaussian Uniqueness)

Theorem 5.2 (Gaussian Uniqueness)

Consider any world satisfying Assumptions 3.1. Suppose every minimizer of (3) with Cov⁡(h⁡(z))=IN\mathrm{Cov}(h(z))=I_{N} is linear, h⁡(z)=Q​zh(z)=Qz. Then zz is Gaussian.

B.1 Background

The converse uses the diffusion generator and its eigenfunctions under the stated stationary density and boundary assumptions.

Equation 30
𝒟​φ=1p⁡(zi)​dd​zi​[K​p​(zi)​φ′​(zi)].\mathcal{D}\varphi=\frac{1}{p(z_{i})}\frac{d}{dz_{i}}\Big[K\,p(z_{i})\,\varphi^{\prime}(z_{i})\Big]. (30)

B.2 Proof

An affine slow eigenfunction forces a linear density score, selecting a Gaussian among the admissible latent laws.

Equations 31–33
𝒟⁡[a​zi+b]=a​K​(log⁡p)′​(zi).\mathcal{D}[az_{i}+b]=a\,K(\log p)^{\prime}(z_{i}). (31)
K​(log⁡p)′​(zi)=−λ​zi−λ​b/a.K(\log p)^{\prime}(z_{i})=-\lambda z_{i}-\lambda b/a. (32)
log⁡p⁡(zi)=−λ2​K​(zi+ba)2+C,\log p(z_{i})=-\frac{\lambda}{2K}\left(z_{i}+\frac{b}{a}\right)^{2}+C, (33)

Remark: the OU transition is a modeling choice.

The Ornstein–Uhlenbeck transition is a chosen world model. Gaussian marginals alone do not force every possible transition to have that form.

Appendix C Proof of Theorem 5.3 (Approximate Identifiability)

Theorem 5.3 (Approximate Identifiability)

Consider the Gaussian world (3.1.1). Let h:ℝN→ℝNh:\mathbb{R}^{N}\to\mathbb{R}^{N} be measurable with 𝔼⁡[h⁡(z)]=0\mathbb{E}[h(z)]=0, satisfying: Approximate alignment: ℒ⁡(h)≤2​(1−ρ)​tr​(Cov⁡(h⁡(z)))+δ\mathcal{L}(h)\leq 2(1-\rho)\,\mathrm{tr}(\mathrm{Cov}(h(z)))+\delta; Approximate whitening: ‖Cov⁡(h⁡(z))−IN‖F≤ε\|\mathrm{Cov}(h(z))-I_{N}\|_{F}\leq\varepsilon. Define D=δ/(2​ρ​(1−ρ))D=\delta/(2\rho(1-\rho)). Then there exists Q∈O⁡(N)Q\in O(N) such that

Equation𝔼⁡[‖h⁡(z)−Q​z‖2]≤D+(ε+D)2\mathbb{E}\!\left[\|h(z)-Qz\|^{2}\right]\;\leq\;D\;+\;(\varepsilon+D)^{2}
(34)

Setup and notation.

The approximate proof names separate alignment and whitening deviations before bounding recovery error.

Equations 35–38
hi​(z)=∑|α|≥1cα(i)​Hα​(z),h_{i}(z)\;=\;\sum_{|\alpha|\geq 1}c_{\alpha}^{(i)}\,H_{\alpha}(z), (35)
Mi​j=𝔼⁡[hi​(z)​zj].M_{ij}\;=\;\mathbb{E}\bigl[h_{i}(z)\,z_{j}\bigr]. (36)
Wnl=𝔼⁡[‖ν⁡(z)‖2].W_{\mathrm{nl}}\;=\;\mathbb{E}\bigl[\|\nu(z)\|^{2}\bigr]. (37)
tr⁡(Cov⁡(h⁡(z)))=‖M‖F2+Wnl.\mathrm{tr}\bigl(\mathrm{Cov}(h(z))\bigr)\;=\;\|M\|_{F}^{2}\;+\;W_{\mathrm{nl}}. (38)

C.1 Step 1: Alignment gap controls nonlinear energy

The spectral gap bounds how much variance can occupy nonlinear components when alignment is nearly optimal.

Equations 39–45
ℒ⁡(h)= 2​N− 2​∑i=1N𝔼⁡[hi​(z′)​hi​(z)],\mathcal{L}(h)\;=\;2N\;-\;2\sum_{i=1}^{N}\mathbb{E}\!\left[h_{i}(z^{\prime})\,h_{i}(z)\right], (39)
𝔼⁡[Hα​(z′)|z]=ρ|α|​Hα​(z),\mathbb{E}\bigl[H_{\alpha}(z^{\prime})\,\big|\,z\bigr]\;=\;\rho^{|\alpha|}\,H_{\alpha}(z), (40)
ℒ⁡(h)= 2​∑i=1N∑|α|≥1(1−ρ|α|)​(cα(i))2.\mathcal{L}(h)\;=\;2\sum_{i=1}^{N}\sum_{|\alpha|\geq 1}(1-\rho^{|\alpha|})\,(c_{\alpha}^{(i)})^{2}. (41)
ℒ⁡(h)≥ 2​(1−ρ)​‖M‖F2+ 2​(1−ρ)​(1+ρ)​Wnl.\mathcal{L}(h)\;\geq\;2(1-\rho)\,\|M\|_{F}^{2}\;+\;2(1-\rho)(1+\rho)\,W_{\mathrm{nl}}. (42)
ℒ⁡(h)\displaystyle\mathcal{L}(h) ≥ 2​(1−ρ)​[tr⁡(Cov⁡(h⁡(z)))−Wnl]+2​(1−ρ)​(1+ρ)​Wnl\displaystyle\;\geq\;2(1-\rho)\bigl[\mathrm{tr}(\mathrm{Cov}(h(z)))-W_{\mathrm{nl}}\bigr]+2(1-\rho)(1+\rho)\,W_{\mathrm{nl}}
= 2​(1−ρ)​tr​(Cov⁡(h⁡(z)))+2​(1−ρ)​[(1+ρ)−1]​Wnl\displaystyle\;=\;2(1-\rho)\,\mathrm{tr}(\mathrm{Cov}(h(z)))+2(1-\rho)\bigl[(1+\rho)-1\bigr]\,W_{\mathrm{nl}}
= 2​(1−ρ)​tr​(Cov⁡(h⁡(z)))+2​ρ​(1−ρ)​Wnl.\displaystyle\;=\;2(1-\rho)\,\mathrm{tr}(\mathrm{Cov}(h(z)))+2\rho(1-\rho)\,W_{\mathrm{nl}}. (43)
ℒ⁡(h)≤ 2​(1−ρ)​tr​(Cov⁡(h⁡(z)))+δ.\mathcal{L}(h)\;\leq\;2(1-\rho)\,\mathrm{tr}(\mathrm{Cov}(h(z)))+\delta. (44)
Wnl≤δ2​ρ​(1−ρ).\boxed{W_{\mathrm{nl}}\;\leq\;\frac{\delta}{2\rho(1-\rho)}.} (45)

C.2 Step 2: Whitening constraint forces M near orthogonal

Covariance error and nonlinear energy bound how far the remaining linear matrix is from an orthogonal matrix.

Equations 46–48
Cov⁡(h⁡(z))=M​M⊤+N,\mathrm{Cov}(h(z))=MM^{\top}+N, (46)
‖M​M⊤−IN‖F=‖Cov⁡(h⁡(z))−IN−N‖F≤ε+Wnl.\|MM^{\top}-I_{N}\|_{F}=\|\mathrm{Cov}(h(z))-I_{N}-N\|_{F}\leq\varepsilon+W_{\mathrm{nl}}. (47)
‖M−Q‖F=‖P−IN‖F≤‖P2−IN‖F≤ε+Wnl.\|M-Q\|_{F}=\|P-I_{N}\|_{F}\leq\|P^{2}-I_{N}\|_{F}\leq\varepsilon+W_{\mathrm{nl}}. (48)

C.3 Step 3: Pythagorean decomposition

Orthogonality separates the two error contributions to obtain the recovery bound.

Equations 49–52
h⁡(z)−Q​z=(M−Q)​z⏟degree 1+ν⁡(z)⏟degree≥2.h(z)-Qz=\underbrace{(M-Q)z}_{\text{degree 1}}+\underbrace{\nu(z)}_{\text{degree}\geq 2}. (49)
𝔼⁡[‖h⁡(z)−Q​z‖2]=𝔼⁡[‖(M−Q)​z‖2]+𝔼⁡[‖ν⁡(z)‖2].\mathbb{E}[\|h(z)-Qz\|^{2}]=\mathbb{E}[\|(M-Q)z\|^{2}]+\mathbb{E}[\|\nu(z)\|^{2}]. (50)
𝔼⁡[‖(M−Q)​z‖2]=tr⁡((M−Q)​𝔼​[z​z⊤]​(M−Q)⊤)=‖M−Q‖F2.\mathbb{E}[\|(M-Q)z\|^{2}]=\mathrm{tr}\!\left((M-Q)\,\mathbb{E}[zz^{\top}]\,(M-Q)^{\top}\right)=\|M-Q\|_{F}^{2}. (51)
𝔼⁡[‖h⁡(z)−Q​z‖2]=‖M−Q‖F2+Wnl≤(ε+Wnl)2+Wnl≤(ε+δ2​ρ​(1−ρ))2+δ2​ρ​(1−ρ).\mathbb{E}[\|h(z)-Qz\|^{2}]=\|M-Q\|_{F}^{2}+W_{\mathrm{nl}}\leq(\varepsilon+W_{\mathrm{nl}})^{2}+W_{\mathrm{nl}}\leq\left(\varepsilon+\frac{\delta}{2\rho(1-\rho)}\right)^{\!2}+\frac{\delta}{2\rho(1-\rho)}. (52)

Role of the Gaussian on the latent side vs. the output side.

Gaussian latent transitions support the spectral argument; approximate output whitening is a different condition and cannot replace the latent-world assumptions.

Appendix D Proof and Discussion of Theorem 5.4

Theorem 5.4 (Optimal Latent Planning)

Let h⁡(z)=Q​zh(z)=Qz with Q∈O⁡(N)Q\in O(N), write z^:=h⁡(z)\hat{z}:=h(z), and consider any finite-horizon optimal control problem whose stage and terminal costs ℓ,ℓT\ell,\ell_{T} are O⁡(N)O(N)-invariant in the state argument:

Equationℓ⁡(R​z,a)=ℓ⁡(z,a),ℓT​(R​z)=ℓT​(z),∀R∈O⁡(N).\ell(Rz,a)=\ell(z,a),\qquad\ell_{T}(Rz)=\ell_{T}(z),\qquad\forall\,R\in O(N).
(53)

Let p^(⋅∣z^t−1,at)\hat{p}(\cdot\mid\hat{z}_{t-1},a_{t}) denote the pushforward of pp under hh, and let V^∗,a^∗1:T\hat{V}^{*},\hat{a}^{*}_{1:T} denote the value function and optimal plan of the z^\hat{z}-space problem defined by replacing pp with p^\hat{p} and leaving costs unchanged. Then V^∗​(h⁡(z0))=V∗​(z0)\hat{V}^{*}\!\big(h(z_{0})\big)=V^{*}(z_{0}) and a^1:T∗(h(z0))=a1:T∗(z0)\hat{a}^{*}_{1:T}\!\big(h(z_{0})\big)=a^{*}_{1:T}(z_{0}).

Equations 54–55
ℓ⁡(z^t,at+1)=ℓ⁡(Q​zt,at+1)=ℓ⁡(zt,at+1)a.s.,ℓT​(z^T)=ℓT​(zT)a.s.\ell(\hat{z}_{t},a_{t+1})\;=\;\ell(Qz_{t},a_{t+1})\;=\;\ell(z_{t},a_{t+1})\quad\text{a.s.},\qquad\ell_{T}(\hat{z}_{T})\;=\;\ell_{T}(z_{T})\quad\text{a.s.} (54)
𝔼p^​[∑t=1Tℓ⁡(z^t−1,at)+ℓT​(z^T)|z^0]=𝔼p​[∑t=1Tℓ⁡(zt−1,at)+ℓT​(zT)|z0].\mathbb{E}_{\hat{p}}\!\left[\sum_{t=1}^{T}\ell(\hat{z}_{t-1},a_{t})+\ell_{T}(\hat{z}_{T})\,\Big|\,\hat{z}_{0}\right]\;=\;\mathbb{E}_{p}\!\left[\sum_{t=1}^{T}\ell(z_{t-1},a_{t})+\ell_{T}(z_{T})\,\Big|\,z_{0}\right]. (55)

D.1 Worked Applications

Goal-reaching.

Euclidean goal-reaching costs are unchanged by the same orthogonal transformation of states and goals.

Linear-quadratic regulation.

Linear-quadratic control transfers by consistently changing state matrices and cost coordinates.

Equations 56–58
z^t+1=(Q​A​Q⊤)​z^t+(Q​B)​at+Q​εt,Q​εt∼𝒩⁡(0,Q​Σ​Q⊤),\hat{z}_{t+1}=(QAQ^{\top})\,\hat{z}_{t}+(QB)\,a_{t}+Q\varepsilon_{t},\qquad Q\varepsilon_{t}\sim\mathcal{N}(0,Q\Sigma Q^{\top}), (56)
P=A⊤​P​A−A⊤​P​B​(R+B⊤​P​B)−1​B⊤​P​A+WP=A^{\top}PA-A^{\top}PB(R+B^{\top}PB)^{-1}B^{\top}PA+W (57)
a^t=−K^​z^t=−(K​Q⊤)​(Q​zt)=−K​zt,\hat{a}_{t}=-\hat{K}\hat{z}_{t}=-(KQ^{\top})(Qz_{t})=-Kz_{t}, (58)

General rotation-invariant objectives.

More general objectives transfer through coordinate transformation; rotation-invariant ones need no changed functional form.

D.2 Scope: encoder versus action-conditioned dynamics

Encoder identifiability does not by itself train the action-conditioned transition or eliminate model error.

D.3 Connection to classical optimal control

The planning statement is a change-of-coordinates result for classical control applied after representation learning.

Appendix E Alternative Proof via Dirichlet Energy

E.1 Setup

The alternative argument studies a small-transition, differentiable regime using Dirichlet energy.

Equation 59
ℒ⁡(h)=limϵ→01ϵ​𝔼​[‖h⁡(z+ϵ​η)−h⁡(z)‖2].\mathcal{L}(h)=\lim_{\epsilon\to 0}\frac{1}{\epsilon}\,\mathbb{E}\!\left[\|h(z+\sqrt{\epsilon}\eta)-h(z)\|^{2}\right]. (59)

E.2 Proof

Theorem 5 (Identifiability via Dirichlet energy)

Any C1C^{1} diffeomorphism hh that preserves the standard Gaussian measure and minimizes ℒ⁡(h)\mathcal{L}(h) must satisfy h⁡(z)=Q​zh(z)=Qz for some orthogonal matrix Q∈O⁡(N)Q\in O(N).

Step 1: Reduction to Dirichlet energy.

A small-step expansion relates paired-state disagreement to a gradient-energy functional.

Equations 60–64
ℒ⁡(h)=𝔼z∼γN​[‖Jh​(z)‖F2],\mathcal{L}(h)=\mathbb{E}_{z\sim\gamma_{N}}\!\left[\|J_{h}(z)\|_{F}^{2}\right], (60)
h⁡(z+ϵ​η)−h⁡(z)=ϵ​Jh​(z)​η+R⁡(z,ϵ​η),h(z+\sqrt{\epsilon}\eta)-h(z)=\sqrt{\epsilon}\,J_{h}(z)\,\eta+R(z,\sqrt{\epsilon}\eta), (61)
‖h⁡(z+ϵ​η)−h⁡(z)‖2=ϵ​‖Jh​(z)​η‖2+2​ϵ​(Jh​(z)​η)⊤​R+‖R‖2.\|h(z{+}\sqrt{\epsilon}\eta)-h(z)\|^{2}=\epsilon\,\|J_{h}(z)\eta\|^{2}+2\sqrt{\epsilon}\,(J_{h}(z)\eta)^{\top}R+\|R\|^{2}. (62)
1ϵ​𝔼z,η​[‖h⁡(z+ϵ​η)−h⁡(z)‖2]→ϵ→0𝔼z,η​[‖Jh​(z)​η‖2].\frac{1}{\epsilon}\,\mathbb{E}_{z,\eta}\!\left[\|h(z{+}\sqrt{\epsilon}\eta)-h(z)\|^{2}\right]\;\xrightarrow{\epsilon\to 0}\;\mathbb{E}_{z,\eta}\!\left[\|J_{h}(z)\eta\|^{2}\right]. (63)
𝔼η​[‖Jh​(z)​η‖2]=tr⁡(Jh​(z)⊤​Jh​(z))=‖Jh​(z)‖F2.\mathbb{E}_{\eta}\!\left[\|J_{h}(z)\eta\|^{2}\right]=\mathrm{tr}(J_{h}(z)^{\top}J_{h}(z))=\|J_{h}(z)\|_{F}^{2}. (64)

Step 2: The expected log-determinant is zero.

The change-of-variables calculation controls the expected Jacobian log determinant under its stated regularity assumptions.

Equations 65–66
ln⁡|detJh​(z)|=12​‖h⁡(z)‖2−12​‖z‖2.\ln|\det J_{h}(z)|=\tfrac{1}{2}\|h(z)\|^{2}-\tfrac{1}{2}\|z\|^{2}. (65)
𝔼⁡[ln⁡|detJh​(z)|]=0.\mathbb{E}\!\left[\ln|\det J_{h}(z)|\right]=0. (66)

Step 3: Lower bound via AM-GM and Jensen.

Arithmetic–geometric mean and convexity inequalities establish the energy lower bound.

Equations 67–69
1N​∑i=1Nσi​(z)2≥(∏i=1Nσi​(z)2)1/N=|detJh​(z)|2/N,\frac{1}{N}\sum_{i=1}^{N}\sigma_{i}(z)^{2}\geq\left(\prod_{i=1}^{N}\sigma_{i}(z)^{2}\right)^{1/N}=|\det J_{h}(z)|^{2/N}, (67)
ℒ⁡(h)≥N​𝔼​[|detJh​(z)|2/N].\mathcal{L}(h)\geq N\,\mathbb{E}\!\left[|\det J_{h}(z)|^{2/N}\right]. (68)
𝔼⁡[|detJh​(z)|2/N]≥exp⁡(2N​𝔼​[ln⁡|detJh​(z)|])=exp⁡(0)=1.\mathbb{E}\!\left[|\det J_{h}(z)|^{2/N}\right]\geq\exp\!\left(\tfrac{2}{N}\,\mathbb{E}\!\left[\ln|\det J_{h}(z)|\right]\right)=\exp(0)=1. (69)

Step 4: Equality forces orthogonal Jacobian.

Equality requires an orthogonal local Jacobian rather than arbitrary nonlinear stretching.

Equation 70
Jh​(z)⊤​Jh​(z)=INfor a.e. ​z.J_{h}(z)^{\top}J_{h}(z)=I_{N}\qquad\text{for a.e.\ }z. (70)

Step 5: Orthogonal Jacobian implies linearity (Mazur–Ulam).

A rigidity argument turns the relevant distance-preserving structure into a global affine/orthogonal map.

Equations 71–73
h⁡(z2)−h⁡(z1)=∫01Jh​(γ⁡(t))​(z2−z1)​𝑑t.h(z_{2})-h(z_{1})=\int_{0}^{1}J_{h}(\gamma(t))(z_{2}-z_{1})\,dt. (71)
‖h⁡(z2)−h⁡(z1)‖≤∫01‖Jh​(γ⁡(t))​(z2−z1)‖​𝑑t=∫01‖z2−z1‖​𝑑t=‖z2−z1‖.\|h(z_{2})-h(z_{1})\|\leq\int_{0}^{1}\|J_{h}(\gamma(t))(z_{2}-z_{1})\|\,dt=\int_{0}^{1}\|z_{2}-z_{1}\|\,dt=\|z_{2}-z_{1}\|. (72)
‖h⁡(z1)−h⁡(z2)‖=‖z1−z2‖for all ​z1,z2∈ℝN,\|h(z_{1})-h(z_{2})\|=\|z_{1}-z_{2}\|\qquad\text{for all }z_{1},z_{2}\in\mathbb{R}^{N}, (73)

Generalization.

The extension is limited by the alternative proof’s differentiability and small-transition conditions.

E.3 Comparison of the Two Proofs

The Hermite proof handles the stated finite-correlation Gaussian setting; the energy proof offers a complementary, more restrictive route.

Hermite proof Dirichlet energy proof
Regularity Measurable C1C^{1} diffeomorphism
Noise level Any ρ∈(0,1)\rho\in(0,1) Infinitesimal (ρ→1\rho\to 1)
Key tool Mehler’s formula AM-GM + Jensen + Mazur–Ulam
Mechanism Low-pass filter on degrees Jacobian rigidity
Connection to Harmonic analysis Differential geometry

Appendix F Prior Work: Connection to Slow Feature Analysis

SFA and its optimization problem.

Slow-feature analysis favors functions that vary least across related states while preventing trivial constants.

Nonlinear identifiability via SFA.

For general worlds, slow features can identify a source only through a nonlinear monotone transformation.

Eigenfunction structure and Sturm–Liouville theory.

Sturm–Liouville structure orders the relevant one-dimensional diffusion eigenfunctions.

The Gaussian specialization and Hermite polynomials.

The Gaussian specialization yields Hermite modes with a tractable ordering by degree.

Sequential versus simultaneous extraction.

Sequentially extracting modes and jointly learning a multivariate representation impose different optimization constraints.

The role of isotropic transitions.

Equal transition scales prevent one coordinate’s nonlinear modes from outranking another coordinate’s linear mode.

Beyond Gaussian latents.

Other latent laws require a different recovery statement; the Gaussian result is not a universal nonlinear-ICA solution.

Relation to Mehler’s formula and maximal correlation.

Mehler’s formula and maximal-correlation facts explain the central preference for linear Gaussian modes.

Practical differences: learned features versus fixed kernels.

A learned neural representation differs from choosing fixed spectral features or a predefined kernel.

Connections to diffusion maps and spectral geometry.

Diffusion-map and spectral-geometry connections supply context for the transition-operator analysis.

Summary of the relationship.

The relationship concerns a shared preference for slow structure, with different assumptions determining what that structure represents.

Sprekeler et al. (2014) This work
Identifiability class Permutation Orthogonal
Latent variable distributions Any independent Gaussian (or i.i.d. factorial)
Transition structure Distinct rates required Isotropic required
Extraction Sequential (greedy) Simultaneous
Function space Fixed polynomial kernel Learned (neural network)
Proof technique Sturm–Liouville / PDE Mehler’s formula / Parseval
Approximate bound None D+(ε+D)2D+(\varepsilon+D)^{2}
Practical algorithm xSFA (fragile, ≤6\leq 6 latent variables) LeJEPA / SIGReg (scalable)
Table 4: Comparison of the SFA identifiability theory of Sprekeler et al. [74] and the present work.

Appendix G Lean Verification

Scope and methodology.

The verification files formalize the paper’s derivations relative to explicitly listed background assumptions.

File / Component Lean name Status
Hermite.lean: Thm. 5.1 via Hermite polynomials
Hermite basis completeness mehler_summability axiomatized
Contraction lemma (ρd\rho^{d} decay) (folded into above) axiomatized
Mehler’s formula correlation_eq_spectral_sum axiomatized
ρd≤ρ\rho^{d}\leq\rho for d≥1d\geq 1 pow_le_self_of_pos_lt_one verified
ρd<ρ\rho^{d}<\rho for d≥2d\geq 2 (strict) pow_lt_self_of_ge_two verified
Pointwise bound wd​ρd≤wd​ρw_{d}\rho^{d}\leq w_{d}\rho spectral_term_le verified
Correlation bound ≤ρ\leq\rho (4) correlation_le_rho verified
Equality ⇔\Leftrightarrow w1=1w_{1}=1 equality_forces_degree_one verified
Degree-1 ⇒\Rightarrow linearity linear_of_degree_one axiomatized
Gaussianity + linear ⇒\Rightarrow orthogonal orthogonal_of_gaussian_linear axiomatized
Loss lower bound ℒ≥2​(1−ρ)​N\mathcal{L}\geq 2(1-\rho)N loss_lower_bound verified
Theorem assembly h=Q​zh=Qz hermite_identifiability verified
Uniqueness.lean: Thm. 5.2 (Gaussian uniqueness)
Affine eigenfunction ⇒\Rightarrow affine score score_affine_of_eigenfunction verified
Affine score ⇒\Rightarrow Gaussian density gaussian_of_affine_score axiomatized
Only-if direction assembly gaussian_of_affine_eigenfunction verified
Gaussian ⇒\Rightarrow Hermite eigenfunctions hermite_first_eigenfunction_of_gaussian axiomatized
Full biconditional gaussian_uniqueness verified
Zero-mean specialization score_pure_linear_zero_mean verified
Dirichlet.lean: Appendix E via Dirichlet energy
AM-GM inequality amgm_sum_ge_prod_pow axiomatized
Jensen’s inequality (exp) exp_mean_ge_mean_exp axiomatized
Mazur–Ulam theorem mazur_ulam axiomatized
Norm-preserving CLM ⇒\Rightarrow isometry clm_isometry_of_norm_preserving verified
Orthogonal Jacobian ⇒\Rightarrow 1-Lipschitz lipschitz_of_orthogonal_jacobian verified
Bilipschitz ⇒\Rightarrow global isometry isometry_of_bilipschitz verified
h⁡(0)=0⇒h(0)=0\Rightarrow linear isometry (part of assembly) verified
Theorem assembly h=Q​zh=Qz dirichlet_identifiability verified
Approx.lean: Thm. 5.3 (approximate bound)
Spectral gap positivity spectral_gap_pos verified
Wnl≤DW_{\mathrm{nl}}\leq D from alignment gap nonlinear_energy_le_D verified
Polar decomposition bound polar_bound_axiom axiomatized
Pythagorean decomposition pythagorean_axiom axiomatized
‖M−Q‖F2\|M-Q\|_{F}^{2} bound linear_deviation_sq_bound verified
Monotonicity of (ε+t)2+t({\varepsilon}+t)^{2}+t bound_monotone verified
Full bound D+(ε+D)2D+(\varepsilon+D)^{2} approximate_identifiability verified
Exact recovery (δ=ε=0\delta=\varepsilon=0) exact_recovery_special_case verified
Planning.lean: Thm. 5.4 (planning equivalence)
Orthogonal invariance definition IsOrthogonalInvariant verified
Stage pushforward stage_pushforward axiomatized
Terminal pushforward terminal_pushforward axiomatized
Per-step stage cost equivalence stage_cost_equiv verified
Terminal cost equivalence terminal_cost_equiv verified
Total cost equivalence (main step) planning_equivalence verified
Minimizer equivalence minimizer_equivalence verified
Value equivalence value_equivalence verified
Table 5: Lean 4 verification status for each proof component. verified: machine-checked with zero sorry. axiomatized: stated as a Lean axiom; standard result not yet in Mathlib or requiring nontrivial API software engineering.

G.1 What Is Axiomatized and Why

1. Hermite polynomial infrastructure.

Hermite-basis completeness and standard polynomial facts are supplied as background infrastructure.

2. Sturm–Liouville spectral theory.

The verification assumes the stated Sturm–Liouville spectral facts from the literature.

3. Standard analysis results requiring API software engineering.

Some standard analysis results are axiomatized where the theorem-prover library interface would require additional work.

4. Measure-theoretic and matrix-analytic facts.

Measure-theoretic and matrix facts are included among the declared background assumptions.

5. Trajectory pushforward for expected costs.

Trajectory pushforward facts support the expected-cost equivalence in planning.

What is verified.

The derived conclusions are checked within that declared formal environment; axiomatized facts are not newly proved by the files.

G.2 Build Information

Build information records the theorem-prover environment needed to reproduce the verification.

Appendix H Experimental Details and Additional Results

H.1 Nonlinear Mixing Functions

The controlled mixing functions are explicitly defined so latent recovery can be measured against known ground truth.

H.2 Network Architecture and Training

Encoder architecture and optimizer settings separate model capacity from the identifiability claim.

Equation 74
ℝ2→Linear​(2,256)→GELU→Linear​(256,256)→GELU→Linear​(256,256)→GELU→Linear​(256,2)ℝ2.\mathbb{R}^{2}\xrightarrow{\text{Linear}(2,256)}\xrightarrow{\text{GELU}}\xrightarrow{\text{Linear}(256,256)}\xrightarrow{\text{GELU}}\xrightarrow{\text{Linear}(256,256)}\xrightarrow{\text{GELU}}\xrightarrow{\text{Linear}(256,2)}\mathbb{R}^{2}. (74)

H.3 Training Procedure

The training loop constructs positive pairs and combines alignment with the selected regularizer.

Equation 75
ℒ=λ​ℒSIG+(1−λ)​ℒinv.\mathcal{L}=\lambda\,\mathcal{L}_{\mathrm{SIG}}+(1-\lambda)\,\mathcal{L}_{\mathrm{inv}}. (75)

H.4 Hyperparameters

The hyperparameter tables define the actual runs and searches rather than a parameter-free training guarantee.

Hyperparameter 2D / Gennorm / Grid Scaling
Optimizer AdamW AdamW
Learning rate 3×10−33\times 10^{-3} 3×10−33\times 10^{-3}
LR schedule warmup (10k) + cosine (10k) warmup (10k) + cosine (10k)
Weight decay 0 0
Batch size 256 256
Training steps 20,000 20,000
Data regime online (infinite) online (infinite)
Eval points 10,000 (fixed) 10,000 (fixed)
Views per sample 2 2
ρ\rho 0.95 (2D/Gennorm), swept (Grid) 0.95
λ\lambda 10−310^{-3} (2D/Gennorm), swept (Grid) 10−610^{-6}
Encoder MLP (spiral/banana/sinusoid) matched inverse-NVP [85]
matched NVP (nvp mixing)
Hidden dimension 256 (MLP) —
MLP layers 4 —
NVP coupling layers 8 (2D/Gennorm) 4
Activation GELU (MLP) tanh (NVP)

H.5 Evaluation Metrics

Metrics distinguish linear recovery, orthogonal recovery, covariance error and planning cost.

H.6 Grid Search

The grid search varies loss balance and transition correlation, exposing optimization-sensitive regimes.

Refer to caption
Refer to caption
Figure 7: Grid search over the regularization weight λ\lambda and OU correlation ρ\rho (5 seeds each, mean ±\pm std). Left: Linear R2R^{2} between true and learned latents (↑\uparrow better). Right: Normalized orthogonality error of the fitted linear map (↓\downarrow better). Identifiability requires both objectives to be active: too much Gaussianity (λ=0.5\lambda=0.5) collapses the representation entirely (R2≈0R^{2}\approx 0), while even weak regularization (λ≤10−2\lambda\leq 10^{-2}) suffices when the alignment signal is strong. At very high correlation (ρ=0.99\rho=0.99), performance degrades for moderate λ\lambda because near-identical positive pairs make the invariance loss trivially small, allowing SIGReg to dominate. The best identifiability (R2>0.96R^{2}>0.96, orthogonality error <0.2<0.2) occurs at λ∈[10−3,10−2]\lambda\in[10^{-3},10^{-2}] with ρ∈[0.8,0.95]\rho\in[0.8,0.95].

H.7 Latent Variable Distribution Sweep Across Mixings

Distribution sweeps test the Gaussian optimum across controlled non-Gaussian alternatives and mixings.

Original figure
Figure 8: Linear identifiability across the gennorm family for all four 2D mixings. R2​(h→z)R^{2}(h\to z) on a fixed evaluation grid as a function of latent shape α\alpha (α=1\alpha{=}1 Laplace, α=2\alpha{=}2 Gaussian). All three methods peak at α=2\alpha=2 as predicted by Thm. 5.2; SIGReg and InfoNCE, which constrain hh beyond second moments, retain a wider plateau than VICReg for heavy-tailed latents. Mean ±\pm std, 3 seeds.
Original figure
Figure 9: Orthogonality error across the gennorm family for all four 2D mixings. ‖Q^⊤​Q^−I‖F/N\|\hat{Q}^{\top}\hat{Q}-I\|_{F}/\sqrt{N} on the same fixed grid (log scale). VICReg and SIGReg dip near α=2\alpha=2 where their constraints align with the latent distribution; InfoNCE remains roughly flat, reflecting weaker control over the linear map under fixed kernel width. Mean ±\pm std over 3 seeds.

H.8 Approximate Bound Decomposition

The bound decomposition compares nonlinear error and covariance distortion with measured recovery.

Refer to caption
Figure 10: Decomposition of the recovery error into its two sources. Left column: Error vs whitening deviation ε\varepsilon. Right column: Error vs alignment gap δ\delta. Top and bottom rows use different marker/color encodings for λ\lambda and ρ\rho. The alignment gap δ\delta is a much stronger predictor of recovery error than the whitening error ε\varepsilon, confirming that approximate whitening is the easy part; the binding constraint is alignment quality.

H.9 Loss Predictivity

Training-loss predictivity is assessed across the stated runs; it remains an empirical checkpoint-selection signal.

SIGReg does not vanish at the optimum.

Finite-sample SIGReg can remain nonzero at an ideal population target, so a residual penalty need not imply equivalent representation error.

Original figure
Figure 11: Losses predict linear identifiability. Each point is one trained encoder from one of our three experiments (2D illustrations, grid search, scaling), zoomed to the converged regime (R2>0.9R^{2}>0.9). Top left: Total loss (alignment + λ\lambda SIGReg) correlates with linear R2​(h→z)R^{2}(h\to z). Top right: Alignment loss alone is predictive of identifiability quality. Bottom left: SIGReg loss vs. R2R^{2}. Bottom right: SIGReg and whitening losses are correlated.

H.10 Scaling Experiment

The scaling study varies latent dimension with matched mixing and encoder capacity.

Table 6: Scaling Experiment (SIGReg) (mean ±\pm std, 5 seeds). The RealNVP mixing is consistently nonlinear across dimensions (R2​(x→z)<1R^{2}(x\to z)<1). The learned model nonetheless recovers the true latents at all dimensions. Training losses are stable; orthogonality error grows gradually with NN. †Batch size 1024, 3 seeds (App. H.10, Tab. 9).
Latents Mixing difficulty Linear identifiability Orthogonality SIGReg losses
NN R2​(z→x)R^{2}(z\to x) R2​(x→z)R^{2}(x\to z) R2​(z→h)R^{2}(z\to h) R2​(h→z)R^{2}(h\to z) ‖Q^⊤​Q^−I‖F/N\|\hat{Q}^{\top}\hat{Q}-I\|_{F}/\sqrt{N} Align SIGReg
±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−7\times 10^{-7} ±\pmstd ×10−7\times 10^{-7} ±\pmstd ×10−5\times 10^{-5} ±\pmstd ×10−4\times 10^{-4} ±\pmstd
212^{1} 0.871±\pm1.4 0.782±\pm2.8 0.99999±\pm3.4 0.99999±\pm3.4 0.000±\pm24 0.0472±\pm15 2.54±\pm1.4
222^{2} 0.865±\pm16 0.724±\pm24 0.99999±\pm12 0.99999±\pm12 0.001±\pm12 0.0475±\pm10 2.09±\pm1.3
232^{3} 0.860±\pm4.2 0.729±\pm10 0.99999±\pm9.0 0.99999±\pm9.0 0.001±\pm14 0.0473±\pm13 2.06±\pm0.78
242^{4} 0.858±\pm6.9 0.733±\pm6.0 0.99998±\pm4.9 0.99998±\pm4.9 0.002±\pm18 0.0476±\pm3.6 1.76±\pm0.18
252^{5} 0.851±\pm3.0 0.738±\pm1.6 0.99998±\pm7.2 0.99998±\pm7.2 0.003±\pm7.6 0.0479±\pm5.0 1.93±\pm0.17
262^{6} 0.854±\pm1.5 0.737±\pm1.4 0.99996±\pm7.6 0.99996±\pm7.4 0.005±\pm10 0.0484±\pm3.9 2.03±\pm0.14
272^{7} 0.853±\pm0.62 0.739±\pm0.50 0.99993±\pm3.1 0.99993±\pm3.2 0.007±\pm7.4 0.0486±\pm1.2 1.80±\pm0.09
282^{8} 0.855±\pm0.30 0.742±\pm0.48 0.99988±\pm7.7 0.99988±\pm7.9 0.010±\pm5.0 0.0486±\pm2.9 1.88±\pm0.17
292^{9} 0.859±\pm0.24 0.749±\pm0.30 0.99977±\pm6.5 0.99977±\pm6.7 0.015±\pm3.7 0.0489±\pm2.4 1.93±\pm0.08
2102^{10} 0.866±\pm0.10 0.763±\pm0.17 0.99956±\pm12 0.99956±\pm12 0.023±\pm4.7 0.0488±\pm0.91 1.93±\pm0.06
2112^{11} 0.869±\pm0.03 0.768±\pm0.05 0.99904±\pm8.1 0.99904±\pm7.9 0.035±\pm1.5 0.0491±\pm0.48 1.88±\pm0.07
212†2^{12\,\dagger} 0.869±\pm0.04 0.768±\pm0.02 0.99945±\pm1.7 0.99944±\pm1.9 0.026±\pm0.90 0.0489±\pm0.14 1.91±\pm0.11
Table 7: Scaling Experiment (VICReg) (mean ±\pm std, 5 seeds). VICReg matches SIGReg on linear identifiability across all dimensions; orthogonality error grows similarly with NN. The whitening loss is stable and small throughout.
Latents Mixing difficulty Linear identifiability Orthogonality VICReg losses
NN R2​(z→x)R^{2}(z\to x) R2​(x→z)R^{2}(x\to z) R2​(z→h)R^{2}(z\to h) R2​(h→z)R^{2}(h\to z) ‖Q^⊤​Q^−I‖F/N\|\hat{Q}^{\top}\hat{Q}-I\|_{F}/\sqrt{N} Align Whitening
±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−7\times 10^{-7} ±\pmstd ×10−7\times 10^{-7} ±\pmstd ×10−5\times 10^{-5} ±\pmstd ×10−4\times 10^{-4} ±\pmstd ×10−4\times 10^{-4}
212^{1} 0.871±\pm0.66 0.782±\pm1.7 0.99999±\pm8.4 0.99999±\pm8.4 0.001±\pm19 0.0491±\pm24 0.0022±\pm13
222^{2} 0.865±\pm15 0.726±\pm24 0.99998±\pm54 0.99998±\pm54 0.001±\pm65 0.0476±\pm11 0.0041±\pm13
232^{3} 0.869±\pm4.4 0.731±\pm8.1 0.99998±\pm4.9 0.99998±\pm4.8 0.002±\pm31 0.0476±\pm13 0.0045±\pm12
242^{4} 0.860±\pm6.9 0.733±\pm5.9 0.99998±\pm4.7 0.99998±\pm4.6 0.002±\pm12 0.0476±\pm3.6 0.0036±\pm2.8
252^{5} 0.855±\pm1.5 0.736±\pm1.9 0.99998±\pm9.4 0.99998±\pm9.4 0.003±\pm3.6 0.0477±\pm6.7 0.0035±\pm3.3
262^{6} 0.853±\pm1.7 0.737±\pm1.4 0.99996±\pm8.2 0.99996±\pm8.1 0.004±\pm9.0 0.0488±\pm2.1 0.0035±\pm1.8
272^{7} 0.853±\pm0.87 0.739±\pm0.66 0.99994±\pm7.1 0.99994±\pm7.2 0.006±\pm9.5 0.0487±\pm0.99 0.0035±\pm0.75
282^{8} 0.855±\pm0.25 0.742±\pm0.49 0.99988±\pm7.2 0.99988±\pm7.2 0.009±\pm4.1 0.0486±\pm2.9 0.0035±\pm0.39
292^{9} 0.858±\pm0.23 0.749±\pm0.30 0.99978±\pm7.2 0.99978±\pm6.9 0.014±\pm3.7 0.0489±\pm2.4 0.0035±\pm0.29
2102^{10} 0.866±\pm0.10 0.763±\pm0.17 0.99958±\pm11 0.99958±\pm11 0.022±\pm4.5 0.0488±\pm0.91 0.0035±\pm0.06
2112^{11} 0.869±\pm0.03 0.768±\pm0.05 0.99909±\pm7.5 0.99909±\pm7.5 0.033±\pm1.3 0.0490±\pm0.47 0.0035±\pm0.11
Table 8: Scaling Experiment (InfoNCE) (mean ±\pm std, 5 seeds). InfoNCE matches the batch-statistic methods at low NN but degrades at scale under a fixed Gaussian kernel width. The InfoNCE loss column shows the saturation directly: at N≥32N\geq 32, both positive- and negative-pair similarities exp(−∥⋅∥2/σ2)\exp(-\|\cdot\|^{2}/\sigma^{2}) underflow to zero, killing the gradient. The non-monotone R2R^{2} across NN (note the spike at N=16N{=}16) is the fingerprint of this kernel-width mismatch rather than a property of the InfoNCE objective itself.
Latents Mixing difficulty Linear identifiability Orthogonality InfoNCE losses
NN R2​(z→x)R^{2}(z\to x) R2​(x→z)R^{2}(x\to z) R2​(z→h)R^{2}(z\to h) R2​(h→z)R^{2}(h\to z) ‖Q^⊤​Q^−I‖F/N\|\hat{Q}^{\top}\hat{Q}-I\|_{F}/\sqrt{N} Align InfoNCE
±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−3\times 10^{-3} ±\pmstd ×10−3\times 10^{-3} ±\pmstd ±\pmstd ×10−3\times 10^{-3} ±\pmstd
212^{1} 0.870±\pm0.92 0.781±\pm1.4 0.96030±\pm2.4 0.95096±\pm1.6 0.505±\pm0.06 0.0653±\pm5.1 4.529±\pm0.03
222^{2} 0.855±\pm11 0.730±\pm23 0.93874±\pm8.1 0.91087±\pm8.2 0.908±\pm0.12 0.0866±\pm5.0 3.557±\pm0.03
232^{3} 0.870±\pm2.0 0.724±\pm12 0.92503±\pm25 0.88681±\pm42 0.876±\pm0.11 0.0883±\pm11 1.811±\pm0.05
242^{4} 0.858±\pm6.4 0.736±\pm6.8 0.99988±\pm0.01 0.99988±\pm0.01 0.010±\pm0.00 0.0489±\pm1.2 0.156±\pm0.01
252^{5} 0.853±\pm2.9 0.737±\pm2.8 0.92206±\pm18 0.90780±\pm26 0.393±\pm0.07 0.0738±\pm6.5 0.000±\pm0.00
262^{6} 0.853±\pm1.7 0.737±\pm1.4 0.77351±\pm0.82 0.64849±\pm3.1 1.756±\pm0.04 0.2123±\pm3.5 0.000±\pm0.00
272^{7} 0.853±\pm0.87 0.739±\pm0.66 0.77232±\pm5.2 0.56695±\pm6.6 4.875±\pm0.08 0.3803±\pm6.2 0.000±\pm0.00
282^{8} 0.855±\pm0.25 0.742±\pm0.49 0.83727±\pm0.51 0.69658±\pm0.49 3.922±\pm0.04 0.2519±\pm2.5 0.000±\pm0.00
292^{9} 0.858±\pm0.23 0.749±\pm0.30 0.84131±\pm0.15 0.70439±\pm0.26 3.946±\pm0.00 0.2507±\pm0.95 0.000±\pm0.00
2102^{10} 0.866±\pm0.10 0.763±\pm0.17 0.84972±\pm0.18 0.72024±\pm0.20 4.003±\pm0.00 0.2507±\pm1.1 0.000±\pm0.00
2112^{11} 0.869±\pm0.03 0.768±\pm0.05 0.85343±\pm0.07 0.72714±\pm0.10 4.028±\pm0.00 0.2514±\pm0.84 0.000±\pm0.00

N = 2 12 .

The largest 4096-dimensional setting reveals a finite-minibatch bottleneck; a larger batch improves the tested estimator.

Table 9: Scaling diagnostic at N=4096N{=}4096. Recovery, orthogonality error, and final alignment loss (mean ±\pm std where more than one seed).
Objective Batch Slices Seeds R2​(h→z)R^{2}(h\to z) ‖Q^⊤​Q^−I‖F/N\|\hat{Q}^{\top}\hat{Q}-I\|_{F}/\sqrt{N} Align
SIGReg 256 256 5 0.663±\pm0.001 2.51±\pm0.15 0.272
SIGReg 256 1024 1 0.663 2.27 0.270
SIGReg 1024 256 3 0.99945±\pm2e-7 0.027 0.049
SIGReg 1024 1024 1 0.99945 0.027 0.049
VICReg 256 – 5 0.652±\pm0.002 2.09±\pm0.09 0.273
InfoNCE 256 – 5 0.727±\pm0.000 4.03±\pm0.00 0.252
Original figure
Figure 12: Scaling with latent dimension (5 seeds, batch size 256). Left: Recovery R2​(h→z)R^{2}(h\to z); open marker: held-out recovery for InfoNCE at N=2048N{=}2048. Right: Normalized orthogonality error of the fitted linear map (log scale), which grows gradually with NN for SIGReg and VICReg. N=4096N{=}4096 is shown in Tab. 9.

H.11 Reacher Experiment

Refer to caption
Figure 13: DMC Reacher. The latent state z=(z0,z1)z=(z_{0},z_{1}) consists of two joint angles (shoulder and wrist) that fully determine the arm configuration. The nonlinear mixing gg is the MuJoCo rendering pipeline producing 64×6464\times 64 pixel observations.

Environment and rendering.

The Reacher renderer is held fixed while the latent-state sampling process changes.

Encoder architecture.

A convolutional encoder maps pixels to the low-dimensional state representation.

Equation
Conv(3→32,4×4,s=2)→BN→GELU→Conv(32→64)→BN→GELU\displaystyle\texttt{Conv}(3\to 32,4\times 4,s{=}2)\to\texttt{BN}\to\texttt{GELU}\to\texttt{Conv}(32\to 64)\to\texttt{BN}\to\texttt{GELU}
→Conv​(64→128)→BN→GELU→Conv​(128→256)→BN→GELU\displaystyle\to\texttt{Conv}(64\to 128)\to\texttt{BN}\to\texttt{GELU}\to\texttt{Conv}(128\to 256)\to\texttt{BN}\to\texttt{GELU}
→AvgPool​(4)→Flatten→Linear​(256,256)→BN→GELU→Linear​(256,2).\displaystyle\to\texttt{AvgPool}(4)\to\texttt{Flatten}\to\texttt{Linear}(256,256)\to\texttt{BN}\to\texttt{GELU}\to\texttt{Linear}(256,2).

Training.

Controlled Gaussian pairs and recorded-trajectory pairs define different training conditions.

Evaluation.

Recovery is measured against known joint coordinates, with planning evaluated separately.

Latent distributions.

The recorded trajectories need not have the independent Gaussian marginals of the controlled condition.

Original figure
Figure 14: Reacher trajectory latent distributions across temporal strides δ\delta. Left: Stationary marginal of (z0,z1)(z_{0},z_{1}); shoulder is broad, wrist is nearly bimodal. Top row: 2D transition differences zt+δ−ztz_{t+\delta}-z_{t}, with the per-dimension R2R^{2} of the best-tuned encoder. Bottom row: Autocorrelation scatter ztz_{t} vs. zt+δz_{t+\delta}, per dimension, with Pearson ρ\rho annotated. Small δ\delta: ρ≈1\rho\approx 1, transition is trivial, alignment carries no signal. Large δ\delta: ρ\rho has dropped but the difference distribution inherits the non-Gaussian structure of the stationary marginals. Intermediate δ\delta: both ρ\rho is informative and the differences are effectively Gaussian, and identifiability is highest.

Gaussianity and autocorrelation jointly determine identifiability.

Both marginal shape and per-coordinate temporal correlation can affect recovery; this experiment changes multiple assumptions.

Additional results on OU and Trajectory conditions.

Additional plots expose the differences between controlled and policy-generated sampling.

Refer to caption
Figure 15: Identifiability requires both approximate Gaussianity and non-trivial autocorrelation. For each temporal stride δ\delta, we plot SIGReg (zscored, averaged over random projections and subsamples, error bars show one standard deviation) against the Pearson autocorrelation ρ\rho of the transition-difference distribution, colored by the corresponding identifiability R2R^{2} (see Table 2). Left: shoulder (z0z_{0}, 1D marginal). Middle: wrist (z1z_{1}, 1D marginal). Right: 2D joint; xx-axis is the average ρ¯\bar{\rho} across dimensions. Dotted red line: SIGReg floor from matched iid 𝒩⁡(0,I)\mathcal{N}(0,I) samples (≈1.2\approx 1.2). Dashed red line: best-performing OU autocorrelation (ρ=0.95\rho=0.95). Small δ\delta (top right): ρ≈1\rho\approx 1 yields no spectral gap, and small differences inherit the non-Gaussian stationary shape; both constraints fail. Large δ\delta (far left): the wrist difference starts resolving the bimodal stationary, so SIGReg climbs again. Intermediate δ∈{8,16}\delta\in\{8,16\}: simultaneously near the Gaussian floor and in a regime of informative ρ\rho; identifiability R2R^{2} peaks here.
Refer to caption
Refer to caption
Figure 16: Left: OU identifiability vs. ρ\rho for different λ\lambda values. R2R^{2} increases monotonically, with all λ\lambda values converging at high ρ\rho. Higher λ\lambda (stronger SIGReg) helps at low ρ\rho where the alignment signal is weak. Right: Gaussian (OU) vs. trajectory data at matched autocorrelation ρ\rho. At the same ρ\rho, Gaussian latents achieve substantially higher R2R^{2}, directly validating the converse theorem: non-Gaussian marginals reduce identifiability.
Refer to caption
Refer to caption
Figure 17: Per-dimension identifiability. Left: OU condition; shoulder and wrist are recovered symmetrically, consistent with the isotropic transition. Right: Trajectory condition; massive asymmetry: the wrist (R2≈0R^{2}\approx 0 at δ=1\delta=1) recovers only at larger δ\delta where temporal variation provides learning signal; the shoulder is consistently easier but degrades at large δ\delta due to wrapping beyond ±π\pm\pi. Per-dimension ρ\rho values are annotated.
Refer to caption
Refer to caption
Figure 18: Left: λ\lambda robustness in the OU condition. For high ρ\rho, identifiability is stable across λ\lambda; for low ρ\rho, stronger regularization (λ=5×10−2\lambda=5\times 10^{-2}) compensates for the weak alignment signal. At ρ=0.99\rho=0.99, the highest λ\lambda degrades slightly as SIGReg begins to dominate alignment. Right: Orthogonality error decreases monotonically with ρ\rho, consistent with the approximate bound (Thm. 5.3).

H.12 Planning Experiment

Refer to caption
Figure 19: Planning: embedding and straight-line paths. Columns: true θ\theta-space (left), Gaussian/OU encoder (center), Trajectory encoder (right). Top row: scatter of eval-set embeddings, colored by true θ\theta-space polar angle. The Gaussian encoder is an approximate rotation of the true latent; the Trajectory encoder is visibly warped. Middle row: three example trajectories that are straight in the true joint space, rendered in each representation. They remain approximately straight in the Gaussian embedding but curve in the Trajectory embedding. Bottom row: the complementary test; straight lines in each model’s latent, decoded back to joint space. Gaussian-encoder plans stay near-straight in θ\theta-space; Trajectory-encoder plans curve. Together, the middle and bottom rows confirm Fig. 5: only linear identifiability turns straight-latent plans into straight joint-space paths.

Setup.

The planning test compares straight latent paths with the corresponding paths in known true-state coordinates.

Metric.

The reported control cost quantifies deviation from the particular goal-reaching optimum.

Interpretation.

Better linear recovery aligns the planning geometry in this task without establishing a universal controller ranking.

H.13 Misspecified Latent Dimension

Setup.

The encoder output dimension is deliberately set below, equal to or above the true latent dimension.

Results.

The reported subspace recovery and covariance participation ratio describe these controlled dimension-mismatch experiments.

Table 10: Misspecified latent dimension (mean over 5 seeds; std ≤0.01\leq 0.01 unless given). PR: participation ratio of Cov⁡(h)\mathrm{Cov}(h). The last MM with PR≈M\mathrm{PR}\approx M (bold) equals the true NN in both settings.
RealNVP, N=8N{=}8 Spiral, N=2N{=}2
MM R2​(h→z)R^{2}(h\to z) R2​(z→h)R^{2}(z\to h) PR     MM R2​(h→z)R^{2}(h\to z) R2​(z→h)R^{2}(z\to h) PR
2 0.242 0.971 2.00     1 0.492 0.985 1.00
4 0.491 0.982 4.00     2 0.985 0.985 2.00
6 0.740 0.986 6.00     3 0.933 0.728 2.83
8 0.987 0.987 7.99     4 0.927 0.637 3.49
12 0.988 0.988 7.99     8 0.976 0.613 3.98
16 0.989 0.989 7.99     16 0.998 0.651 3.80
32 0.949±\pm0.06 0.988 7.19±\pm0.75      
64 0.977±\pm0.02 0.986 7.71±\pm0.39      
Original figure
Figure 20: Misspecified latent dimension, spiral mixing (true N=2N{=}2, 5 seeds; compare Fig. 6). Left: Recovery tracks M/NM/N below NN and saturates above; unlike RealNVP, encoding R2​(z→h)R^{2}(z\to h) drops for M>NM>N because the surplus dimensions carry nonlinear functions of zz. Right: At M=1M{=}1, the encoder keeps one latent variable and discards the other.

H.14 Predictors

Setup.

Linear and multilayer predictors are added to test an extension outside the simplest alignment formulation.

Results.

Recovery remains strong in the tested runs, but a general expressive-predictor guarantee is not established.

Table 11: Predictors. Recovery R2​(h→z)R^{2}(h\to z), mean ±\pm std over 5 seeds.
Predictor RealNVP (N=8N{=}8) Spiral (N=2N{=}2)
None (Eq. 3) 0.989±\pm0.001 0.989±\pm0.000
Linear 0.987±\pm0.001 0.989±\pm0.000
MLP 0.988±\pm0.001 0.988±\pm0.000
Original figure
Figure 21: Predictors. Left: Recovery for the three arms (mean ±\pm std, 5 seeds). Right: Distance of the linear predictor from the true transition operator, ‖P−ρ​I‖F\|P-\rho I\|_{F}, during training (lines are seeds).

H.15 Training Dynamics

Recovery can appear after an initial optimization plateau. The population theorem does not specify when training must reach it.

Table 12: Escape step, the first evaluation step with R2​(h→z)>0.9R^{2}(h\to z)>0.9 (median and range over 5 seeds; SIGReg scaling runs, batch size 256).
NN 2 4 8 16 32 64 128 256 512 1024 2048
median 500 800 1500 1500 1500 1000 700 600 600 600 1000
range 200–600 600–1500 900–3500 700–2000 800–3000 700–3000 600–800 600 500–600 600 1000
Original figure
Figure 22: Recovery during training (SIGReg scaling runs, seed 0, N≤1024N\leq 1024). Triangles mark the escape step, where R2​(h→z)R^{2}(h\to z) first exceeds 0.90.9 (dashed line).

H.16 ImageNet-X Linear Probes

Encoders.

The visual-encoder comparison records architectures, pretraining data and schedules.

Analysis space.

The chosen feature space determines which annotated factors the downstream probes can access.

Probes.

Linear probes measure labeled ImageNet-X factors without finetuning the visual encoder.

Results.

The results compare factor-readout accuracy under the stated protocols, not recovery of a known full natural-image generative state.

Gaussianity.

Empirical Gaussianity is approximate. Natural-image comparisons therefore remain empirical evidence beyond the controlled proof setup.

Table 13: ImageNet-X linear probe accuracy, all 17 factors (balanced accuracy, mean over 5 repetitions; chance 0.50.5). NN: balanced sample size. Bold: above chance at a Bonferroni-corrected p<0.05p<0.05 over all factors and models. Factors above the line (N>1000N>1000) enter the main-text mean.
IN-1k, matched reference controls
Factor NN LeJEPA DINO iBOT DINOv2 DINOv3 init pixels
class 43500 0.865 0.666 0.741 0.856 0.813 0.620 0.585
shape 1276 0.737 0.590 0.661 0.709 0.657 0.552 0.575
subcategory 1118 0.720 0.604 0.648 0.682 0.614 0.554 0.502
pattern 11354 0.674 0.593 0.618 0.658 0.653 0.536 0.536
smaller 2738 0.651 0.618 0.663 0.627 0.692 0.529 0.538
partial view 1208 0.629 0.543 0.592 0.595 0.606 0.507 0.506
color 11642 0.616 0.571 0.580 0.599 0.606 0.544 0.526
pose 28404 0.605 0.559 0.583 0.598 0.589 0.530 0.516
background 27898 0.594 0.559 0.570 0.576 0.578 0.532 0.517
darker 230 0.809 0.714 0.751 0.643 0.620 0.700 0.520
person blocking 112 0.800 0.647 0.771 0.671 0.712 0.482 0.588
texture 542 0.664 0.619 0.643 0.698 0.625 0.548 0.521
larger 284 0.614 0.551 0.549 0.493 0.558 0.528 0.493
style 84 0.600 0.562 0.638 0.500 0.608 0.438 0.500
brighter 84 0.592 0.523 0.485 0.492 0.523 0.562 0.562
multiple objects 76 0.555 0.445 0.627 0.491 0.536 0.473 0.491
object blocking 152 0.552 0.565 0.435 0.574 0.513 0.491 0.474
mean (N>1000N>1000) 0.677 0.589 0.628 0.656 0.645 0.545 0.533
mean (all 17) 0.663 0.584 0.621 0.615 0.618 0.537 0.526
Table 14: ImageNet-X probe summary and Gaussianity diagnostics in the 64-dimensional analysis space. EP: sketched Epps–Pulley distance to 𝒩⁡(0,I)\mathcal{N}(0,I) divided by the sample size (lower is more Gaussian). PR/K/K: participation ratio of the covariance divided by the dimension (11 for isotropic). For LeJEPA the coordinates are the projector output; for all other rows, principal components.
Linear MLP Gap EP PR/K/K Excess kurtosis
LeJEPA (projector) 0.663 0.684 0.021 0.0049 0.91 0.39
LeJEPA backbone (768-d) – – – 0.2534 0.08 0.38
DINO 0.584 0.617 0.033 0.5330 0.99 0.04
iBOT 0.621 0.632 0.011 0.6083 0.98 0.03
DINOv2 0.615 0.645 0.030 0.4712 0.92 0.82
DINOv3 0.618 0.643 0.025 0.0033 0.94 0.65
init 0.537 0.554 0.017 0.5311 0.06 1.18
pixels 0.526 0.540 0.014 0.5874 0.14 0.99
𝒩⁡(0,I)\mathcal{N}(0,I) sample – – – 0.00003 1.00 0.00
Concept demonstration

Open image ↗