Literature
Display
← All papers

LeHami: Symplectic Latent Dynamics for World Models

Romina Yalovetzky · Randall Balestriero · Yann LeCun

Preprint posted · arXiv (preprint) ↗

World models9 min text · CondensedOriginal paper ↗
Contents

Overview

The authors release code and a project website for their structured latent world model.

At a glance

Intuition: an ordinary learned predictor can repeatedly squeeze different states together until a long imagined trajectory loses motion information. LeHami replaces that predictor with a learned Hamiltonian and a volume-preserving leapfrog update. Its latent state has position-like and momentum-like halves, with the latter estimated from recent encoded frames. The construction prevents particular rollout-collapse mechanisms and supports changing the integration step. It does not guarantee accurate predictions, prevent encoder collapse or recover the true physical Hamiltonian. Experiments use conservative and driven simulated pendulums. LeHami improves several velocity-prediction and compute measures, while a baseline with a suitably asymmetric temporal encoder achieves higher goal-reaching success.

1 Introduction

The paper separates encoder collapse during learning from contraction caused by repeatedly applying a predictor. Training regularization can address the first without preventing the second. Hamiltonian structure constrains every deployed transition. Position and motion also require different evidence: one image may show where a pendulum is while leaving its direction of travel ambiguous.

Refer to caption
Figure 1: LeHami: Learning symplectic world models end-to-end. Bottom left: Actuated pendulum frames and applied torque sequence utu_{t} (green). Upper left: Latent phase space (𝐪t,𝐩t)(\mathbf{q}_{t},\mathbf{p}_{t}), where transitions are generated by a learned scalar Hϕ​(𝐪t,𝐩t,ut)=Kϕ​(𝐩t)+Vϕ​(𝐪t,ut)H_{\phi}(\mathbf{q}_{t},\mathbf{p}_{t};u_{t})=K_{\phi}(\mathbf{p}_{t})+V_{\phi}(\mathbf{q}_{t},u_{t}) via a symplectic step. Grey lines indicate level sets; the blue curve shows the rollout. Right: Three properties of the Hamiltonian flow: (i) Volume preservation: Latent volume is exactly preserved under any action sequence. (ii) Continuous step size: HϕH_{\phi} integrates at arbitrary time intervals, including between observed frames. (iii) Time-reversibility: Integrates backward by inverting time (t→−tt\rightarrow-t) and reversing actions.

Contributions.

The contributions are a symplectic predictor, a fixed derivative channel, analysis of the collapse modes excluded by the update, and pendulum experiments on forecasting and planning. Architectural guarantees concern the latent map. Empirical physical readouts are needed to assess what the learned state actually contains.

2 LeHami: Symplectic Hamiltonian JEPA

Imagine a small cloud of starting states. A symplectic update can stretch and shear it but preserves its volume. With the stated smoothness and fixed-action-within-step conditions, leapfrog is invertible and cannot uniformly shrink every direction or merge distinct latent states. Volume preservation allows some directions to contract if others expand; it supplies no bound on prediction error or numerical stability at a large step size. The original formal statements are retained below.

Proposition 1 (Symplectic volume preservation and the rollout failure modes it removes).

Let Ψh\Psi_{h} be the leapfrog step of Eq. 4 for step size hh generated by a separable Hϕ=Kϕ1​(𝐩)+Vϕ2​(𝐪,𝐮)H_{\phi}=K_{\phi_{1}}(\mathbf{p})+V_{\phi_{2}}(\mathbf{q},\mathbf{u}) with 𝐮\mathbf{u} held fixed within the step. Then Ψh\Psi_{h} is an exactly symplectic diffeomorphism of ℝ2​d\mathbb{R}^{2d} for every step size hh and every parameter ϕ\phi; it then preserves phase-space volume exactly at every horizon. Here hh is the integrator’s step, which equals the observation interval Δ​t\Delta t during training but need not at inference (Section 3.3). Consequently, three collapse modes are ruled out: uniform contraction, convergence to an asymptotically stable fixed point, and the merging of distinct states. They are stated and proved in Appendix C.1.

Original figure
Figure 2: LeHami multi-step training. Left: at the anchor, the encoder block (Enc + SG) maps a window of W+1W+1 frames, drawn here as the causal window 𝐱t−W,…,𝐱t\mathbf{x}_{t-W},\dots,\mathbf{x}_{t}, to the latent state (𝐪t,𝐩t)(\mathbf{q}_{t},\mathbf{p}_{t}). The Hamiltonian flow (Ham flow) advances it by one leapfrog step of the separable Hamiltonian Hϕ=Kϕ1​(𝐩)+Vϕ2​(𝐪,𝐮)H_{\phi}=K_{\phi_{1}}(\mathbf{p})+V_{\phi_{2}}(\mathbf{q},\mathbf{u}), with the action entering the potential, and is applied KK times to its own outputs. Middle: each prediction (𝐪~t+k,𝐩~t+k)(\tilde{\mathbf{q}}_{t+k},\tilde{\mathbf{p}}_{t+k}) is scored by the prediction loss (C) against the encoding of the corresponding frames, shown here for the last step k=Kk=K. The regularizer (R) acts on the encoder’s embeddings of 𝐪\mathbf{q} (blue) and 𝐩\mathbf{p} (orange). Right: the encoder block as instantiated in this work: a pixel encoder (Enc) outputs 𝐪t\mathbf{q}_{t}, and a fixed Savitzky–Golay stencil (SG) derives 𝐩t\mathbf{p}_{t} from 𝐪t−3,…,𝐪t\mathbf{q}_{t-3},\dots,\mathbf{q}_{t}.

2.1 The architecture

The encoder outputs position-like features. A fixed Savitzky–Golay derivative of recent features initializes the momentum-like half. Two neural networks define a separable scalar Hamiltonian: one depends on momentum, the other on position and action. A kick–drift–kick update advances the pair, and multi-step prediction losses train it against encoded trajectories. SIGReg still shapes encoder features. After initialization, momentum evolves inside the predictor rather than being recomputed from future images.

Equations 1–5
Encoder + SG:\displaystyle\text{Encoder + SG:} 𝐪t=Encθ(𝐱𝒲⁡(t)),𝐩t=SG(𝐪t−3,…,𝐪t),\displaystyle\mathbf{q}_{t}=\text{Enc}_{\theta}(\mathbf{x}_{\mathcal{W}(t)}),\qquad\mathbf{p}_{t}=\text{SG}(\mathbf{q}_{t-3},\dots,\mathbf{q}_{t}), (1)
Hamiltonian flow:\displaystyle\text{Hamiltonian flow:} (𝐪~t+1,𝐩~t+1)=Ham flowϕ​(𝐪t,𝐩t,𝐮t),\displaystyle(\tilde{\mathbf{q}}_{t+1},\tilde{\mathbf{p}}_{t+1})=\text{Ham flow}_{\phi}(\mathbf{q}_{t},\mathbf{p}_{t},\mathbf{u}_{t}),
𝐪˙=∂Hϕ∂𝐩,𝐩˙=−∂Hϕ∂𝐪,\dot{\mathbf{q}}=\frac{\partial H_{\phi}}{\partial\mathbf{p}},\qquad\dot{\mathbf{p}}=-\frac{\partial H_{\phi}}{\partial\mathbf{q}}, (2)
Hϕ​(𝐪,𝐩,𝐮)=Kϕ​(𝐒−1​𝐩)+Vϕ​(𝐪,𝐮),𝐒=diag⁡(e𝝈),H_{\phi}(\mathbf{q},\mathbf{p},\mathbf{u})=K_{\phi}\!\left(\mathbf{S}^{-1}\mathbf{p}\right)+V_{\phi}(\mathbf{q},\mathbf{u}),\qquad\mathbf{S}=\operatorname{diag}\!\left(e^{\bm{\sigma}}\right), (3)
𝐩~t+12\displaystyle\tilde{\mathbf{p}}_{t+\frac{1}{2}} =𝐩t−Δ​t2​∇𝐪Vϕ​(𝐪t,𝐮t),\displaystyle=\mathbf{p}_{t}-\tfrac{\Delta t}{2}\,\nabla_{\mathbf{q}}V_{\phi}(\mathbf{q}_{t},\mathbf{u}_{t}), (4)
𝐪~t+1\displaystyle\tilde{\mathbf{q}}_{t+1} =𝐪t+Δt𝐒−1∇Kϕ(𝐒−1𝐩~t+12),\displaystyle=\mathbf{q}_{t}+\Delta t\,\mathbf{S}^{-1}\,\nabla K_{\phi}\!\left(\mathbf{S}^{-1}\tilde{\mathbf{p}}_{t+\frac{1}{2}}\right),
𝐩~t+1\displaystyle\tilde{\mathbf{p}}_{t+1} =𝐩~t+12−Δ​t2​∇𝐪Vϕ​(𝐪~t+1,𝐮t),\displaystyle=\tilde{\mathbf{p}}_{t+\frac{1}{2}}-\tfrac{\Delta t}{2}\,\nabla_{\mathbf{q}}V_{\phi}(\tilde{\mathbf{q}}_{t+1},\mathbf{u}_{t}),
ℒ\displaystyle\mathcal{L} =1K​∑j=1K‖𝐪~A+j−𝐪A+j‖2+λp​1K​∑j=1K‖𝐩~A+j−𝐩A+j‖2+λ𝒮​𝒮,\displaystyle=\frac{1}{K}\sum_{j=1}^{K}\big\|\tilde{\mathbf{q}}_{A+j}-\mathbf{q}_{A+j}\big\|^{2}+\lambda_{p}\,\frac{1}{K}\sum_{j=1}^{K}\big\|\tilde{\mathbf{p}}_{A+j}-\mathbf{p}_{A+j}\big\|^{2}+\lambda_{\mathcal{S}}\mathcal{S}, (5)

2.2 Related work

Related models also impose mechanical structure, but differ in whether they reconstruct images, add dissipative dynamics, learn a separate control port or use a non-symplectic integrator. The same H-JEPA label appears in an independent Hamiltonian paper; it should not be confused with the hierarchical H-JEPA edition in this collection.

3 Empirical validation of LeHami

Two simulated pendulum systems test the model: an unforced system and one driven by recorded torques. Both preserve physical phase-space volume, matching the chosen structural prior. Rod lengths vary, and the driven system also varies mass. These are favorable mechanical testbeds rather than evidence for arbitrary dissipative environments.

3.1 Experimental setup

The main comparison shares training data, encoder family and learning setup with LeWM, changing the predictor and adding the derivative channel. Frozen probes read angle and angular velocity from features. Using a probe fitted after one predictor step gives LeWM access to temporal context; anchor-only comparisons otherwise favor a state with an explicit history-derived motion channel. Planning uses CEM with feedback replanning.

Refer to caption
Figure 3: The two pendulum environments. Top: the conservative pendulum, with no actuation. Bottom: the actuated pendulum. Beneath each row is its respective torque utu_{t}, drawn as the piecewise-constant signal, held constant for three frames.

3.2 Evaluating Complete Phase-Space Physics Capture

LeHami retains useful velocity information and avoids measured latent-volume loss. Some angle-prediction results favor LeWM, especially early in the driven rollout. The derivative channel contributes information unavailable to a single-frame encoder. These results support the combined state construction and predictor; they do not prove that symplecticity alone recovers every physical quantity.

Refer to caption
Figure 4: Prediction performance during conservative pendulum rollouts. Wrapped MAE of θ\theta (left) and ω\omega (middle) across the test rollout steps (KK). Metrics are computed using a kk-NN probe (k=25k=25) fitted at K=1K=1 and kept frozen. Shaded areas indicate variance across three training seeds, averaged over test episodes. Cumulative log-volume of the one-step latent map, averaged over test episodes, for one training seed (right).
Table 1: Performance in physics prediction of the actuated pendulum at the anchor K=0. Mean ±\pm s.d.over three training seeds, test split. Bold marks the better model within a probe tier. A negative R2R^{2} is below the predict-the-mean floor.
model probe MAE θ\theta [10−210^{-2} rad] MAE ω\omega [rad/s] R2​(ω)R^{2}(\omega)
LeHami ridge 23.0±2.023.0\pm 2.0 1.168±0.001\mathbf{1.168\pm 0.001} +0.491±0.005\mathbf{+0.491\pm 0.005}
25-NN 6.8±1.86.8\pm 1.8 0.419±0.045\mathbf{0.419\pm 0.045} +0.889±0.024\mathbf{+0.889\pm 0.024}
MLP 2.9±0.42.9\pm 0.4 0.184±0.019\mathbf{0.184\pm 0.019} +0.987±0.004\mathbf{+0.987\pm 0.004}
LeWM ridge 1.4±0.1\mathbf{1.4\pm 0.1} 2.422±0.0012.422\pm 0.001 −0.014±0.001-0.014\pm 0.001
25-NN 1.1±0.0\mathbf{1.1\pm 0.0} 2.508±0.0022.508\pm 0.002 −0.103±0.001-0.103\pm 0.001
MLP 1.9±0.1\mathbf{1.9\pm 0.1} 2.419±0.0012.419\pm 0.001 −0.011±0.001-0.011\pm 0.001

3.3 LeHami is a world model with time re-stepping and reversal

The learned vector field can be integrated at smaller sub-frame steps, and forward/backward integration has small numerical round-trip error. Doubling the step breaks the demonstrated prediction. Symplecticity is an exact algebraic property of the update, while prediction accuracy and stability remain step-size dependent. Reversing an action-conditioned rollout also requires reversing the applied action schedule.

Refer to caption
Figure 5: The learned flow behaves like a continuous-time vector field, and is approximately reversible, showcased with the conservative pendulum. Left: MAE of θ\theta integrated with step sizes never seen in training (to the ground-truth from the integration of the ODE). The model is trained only at Δ​t=0.1\Delta t=0.1 s; halving and quartering the step (2 and 4 sub-steps per frame, respectively) traces the same trajectory, while doubling it breaks. Right: Round-trip error after rolling KK steps out and back by sign flip Δ​t→−Δ​t\Delta t\to-\Delta t in the HϕH_{\phi}.

3.4 LeHami brings computational efficiency and structural priors in velocity planning

Under the reported fifteen-TFLOP episode budget, the cheaper predictor permits more frequent replanning. LeHami captures velocity direction better than single-frame or time-symmetric baselines. An untied temporal encoder nevertheless achieves the strongest success rates, and LeHami’s joint position–velocity success is limited by reaching the target angle. The claim is a useful structural and compute tradeoff, not universal planning dominance.

Table 2: Goal-reaching success on the actuated pendulum, by clause. Best success probability per model at a matched budget of at most 1515 TFLOP/episode. Mean ±\pm s.d. over three model seeds. The floor probabilities (corresponding to u=0u=0) are 0.0600.060, 0.2400.240 and 0.0400.040 on each of the three clauses respectively. Bold marks the better model within a clause. The planner re-solves every RR steps towards the same goal at horizon 30, so it makes ⌈30/R⌉\lceil 30/R\rceil solves, the last of which covers only the remaining steps.
Clause Model Success TFLOP Replanning interval
Position LeWM W=1W{=}1 0.800±0.019\mathbf{0.800\pm 0.019} 12.87 R=9R=9, 4 solves
LeWM W=3W{=}3 (tied) 0.633±0.0200.633\pm 0.020 13.65 R=9R=9, 4 solves
LeWM W=3W{=}3 (untied) 0.696±0.0160.696\pm 0.016 13.65 R=9R=9, 4 solves
LeHami 0.480±0.0840.480\pm 0.084 8.05 R=9R=9, 4 solves
Velocity LeWM W=1W{=}1 0.184±0.0270.184\pm 0.027 8.78 R=15R=15, 2 solves
LeWM W=3W{=}3 (tied) 0.473±0.0140.473\pm 0.014 13.65 R=9R=9, 4 solves
LeWM W=3W{=}3 (untied) 0.800±0.058\mathbf{0.800\pm 0.058} 13.65 R=9R=9, 4 solves
LeHami 0.527±0.0190.527\pm 0.019 10.98 R=6R=6, 5 solves
Position and Velocity LeWM W=1W{=}1 0.142±0.0330.142\pm 0.033 12.87 R=9R=9, 4 solves
LeWM W=3W{=}3 (tied) 0.400±0.0370.400\pm 0.037 9.31 R=15R=15, 2 solves
LeWM W=3W{=}3 (untied) 0.636±0.025\mathbf{0.636\pm 0.025} 13.65 R=9R=9, 4 solves
LeHami 0.304±0.0880.304\pm 0.088 10.98 R=6R=6, 5 solves

4 Conclusion

The study shows how a mechanical constraint can prevent particular latent rollout failures and supply a useful motion-aware state. The guarantee and the empirical advantage have different scopes: the map preserves volume by design, while physical accuracy and planning quality depend on learning, representation and task.

Limitations and future work.

Strong friction contracts physical phase space and conflicts with the pure volume-preserving prior. Configuration-dependent mass also challenges the fixed derivative-as-momentum construction. Dissipative extensions and learned momentum states are proposed directions, not evaluated solutions. Static pose tasks may benefit less from this extra structure.

AI use statement

The source discloses AI assistance with writing, code and proof checking, while retaining author responsibility for the scientific content.

Ethics statement

The authors describe the work as foundational research without a specific immediate societal application.

Acknowledgements

The source acknowledges guidance and feedback on the research and experiments.

Reproducibility statement

The release records configurations, seeded data generators, episode splits, checkpoint selection and evaluation scripts. Reproduction should follow those artifacts, especially derivative windows, probe fitting and the planning budget. Some supplementary comparisons use one seed, unlike the main three-seed results.

Appendix A Appendix organization

The appendices separate mathematical guarantees, architectural choices, data generation, evaluation and supplementary results. This distinction is useful: a proved map property is not the same as a measured physical capability.

Appendix B Background in Hamiltonian mechanics

Hamiltonian dynamics evolve a paired position–momentum state using derivatives of one scalar function. In physical mechanics that function is energy; here the learned scalar mainly supplies a geometric restriction on latent evolution. It need not equal the environment’s physical energy.

Equation 6
𝐪˙=∂H∂𝐩,𝐩˙=−∂H∂𝐪.\dot{\mathbf{q}}=\frac{\partial H}{\partial\mathbf{p}},\qquad\dot{\mathbf{p}}=-\frac{\partial H}{\partial\mathbf{q}}. (6)

Phase-space density.

Following a cloud of initial conditions under Hamiltonian flow preserves its density along motion. This is an incompressibility statement in the full phase space, not a claim that each coordinate or projected picture stays unchanged.

Symplectic form.

The symplectic form tracks oriented areas across paired coordinates. Preserving this structure implies full phase-space volume preservation. Projected areas or individual distances can still change.

Appendix C Theory

C.1 Symplecticity of the leapfrog step: formal statement and proof

The proof decomposes leapfrog into invertible momentum and position shears. Their symmetric Hessians make each shear symplectic; composition retains that property. With twice continuously differentiable generator terms and an action held fixed within each step, the Jacobian determinant is one. Thus distinct states remain distinct, uniform contraction is impossible and a fixed point cannot attract an open neighborhood. This is exact-arithmetic structure; finite precision introduces numerical error.

Equation
𝐉=(𝟎𝐈d−𝐈d𝟎)∈ℝ2​d×2​d,𝐉⊤=−𝐉=𝐉−1.\mathbf{J}\;=\;\begin{pmatrix}\mathbf{0}&\mathbf{I}_{d}\\ -\mathbf{I}_{d}&\mathbf{0}\end{pmatrix}\in\mathbb{R}^{2d\times 2d},\qquad\mathbf{J}^{\!\top}=-\mathbf{J}=\mathbf{J}^{-1}.
Proposition 2 (Formal form of Proposition 1).

Let Kϕ1∈C2​(ℝd)K_{\phi_{1}}\in C^{2}(\mathbb{R}^{d}) and Vϕ2∈C2​(ℝd×𝒰)V_{\phi_{2}}\in C^{2}(\mathbb{R}^{d}\times\mathcal{U}), and let Ψh\Psi_{h} be the leapfrog step of Eq. equation 4 generated by Hϕ=Kϕ1​(𝐩)+Vϕ2​(𝐪,𝐮)H_{\phi}=K_{\phi_{1}}(\mathbf{p})+V_{\phi_{2}}(\mathbf{q},\mathbf{u}) with the control 𝐮\mathbf{u} held fixed within the step. This means that the leapfrog step can be written as Ψh=Ah/2∘Bh∘Ah/2\Psi_{h}=A_{h/2}\circ B_{h}\circ A_{h/2} with

Equation 7
Ah/2​(𝐪,𝐩)=(𝐪,𝐩−h2​∇𝐪Vϕ2​(𝐪,𝐮)),Bh​(𝐪,𝐩)=(𝐪+h​∇𝐩Kϕ1​(𝐩),𝐩).A_{h/2}(\mathbf{q},\mathbf{p})=\bigl(\mathbf{q},\;\mathbf{p}-\tfrac{h}{2}\,\nabla_{\mathbf{q}}V_{\phi_{2}}(\mathbf{q},\mathbf{u})\bigr),\qquad B_{h}(\mathbf{q},\mathbf{p})=\bigl(\mathbf{q}+h\,\nabla_{\mathbf{p}}K_{\phi_{1}}(\mathbf{p}),\;\mathbf{p}\bigr). (7)

Then Ψh\Psi_{h} is a C1C^{1} diffeomorphism of ℝ2​d\mathbb{R}^{2d} that is exactly symplectic, which means that

Equation 8
D​Ψh​(𝐳)⊤​𝐉​D​Ψh​(𝐳)=𝐉for every ​𝐳∈ℝ2​d,D\Psi_{h}(\mathbf{z})^{\!\top}\,\mathbf{J}\,D\Psi_{h}(\mathbf{z})\;=\;\mathbf{J}\qquad\text{for every }\mathbf{z}\in\mathbb{R}^{2d}, (8)

for every step size h>0h>0, every control 𝐮∈𝒰\mathbf{u}\in\mathcal{U} and every parameter ϕ=(ϕ1,ϕ2)\phi=(\phi_{1},\phi_{2}). Consequently detD​Ψh≡1\det D\Psi_{h}\equiv 1, Lebesgue measure volume on ℝ2​d\mathbb{R}^{2d} is Ψh\Psi_{h}-invariant, and for every horizon K∈ℕK\in\mathbb{N} the three below conditions are met:

  1. (i)

    (no uniform contraction) the singular values of D⁡(ΨhK)​(𝐳)D(\Psi_{h}^{K})(\mathbf{z}) with ΨhK:=Ψh∘⋯∘Ψh\Psi_{h}^{K}:=\Psi_{h}\circ\cdots\circ\Psi_{h} satisfy ∏i=12​dσi=1\prod_{i=1}^{2d}\sigma_{i}=1, hence σmax≥1≥σmin\sigma_{\max}\geq 1\geq\sigma_{\min} at every 𝐳∈ℝ2​d\mathbf{z}\in\mathbb{R}^{2d};

  2. (ii)

    (no asymptotically stable fixed point) Ψh\Psi_{h} admits no asymptotically stable fixed point, where “asymptotically stable” means: a fixed point 𝐳∗\mathbf{z}^{\ast} that is Lyapunov stable and attracts every point of some open neighbourhood of itself;

  3. (iii)

    (no merging) ΨhK\Psi_{h}^{K} is a composition of diffeomorphisms, hence itself a diffeomorphism of ℝ2​d\mathbb{R}^{2d}, in particular injective. Consequently, if 𝐳1≠𝐳2\mathbf{z}_{1}\neq\mathbf{z}_{2} then ΨhK​(𝐳1)≠ΨhK​(𝐳2)\Psi_{h}^{K}(\mathbf{z}_{1})\neq\Psi_{h}^{K}(\mathbf{z}_{2}) for every K∈ℕK\in\mathbb{N}: two latents that differ at the anchor remain distinct at every horizon.

Equations
D​Ah/2​(𝐪,𝐩)=(𝐈d𝟎−h2​∇𝐪𝐪2Vϕ2​(𝐪,𝐮)𝐈d),D​Bh​(𝐪,𝐩)=(𝐈dh​∇𝐩𝐩2Kϕ1​(𝐩)𝟎𝐈d),DA_{h/2}(\mathbf{q},\mathbf{p})=\begin{pmatrix}\mathbf{I}_{d}&\mathbf{0}\\ -\tfrac{h}{2}\,\nabla^{2}_{\mathbf{q}\mathbf{q}}V_{\phi_{2}}(\mathbf{q},\mathbf{u})&\mathbf{I}_{d}\end{pmatrix},\qquad DB_{h}(\mathbf{q},\mathbf{p})=\begin{pmatrix}\mathbf{I}_{d}&h\,\nabla^{2}_{\mathbf{p}\mathbf{p}}K_{\phi_{1}}(\mathbf{p})\\ \mathbf{0}&\mathbf{I}_{d}\end{pmatrix},
ΨhK​(C)⊆Bε​(𝐳∗)for all ​K≥Kε.\Psi_{h}^{K}(C)\subseteq B_{\varepsilon}(\mathbf{z}^{\ast})\qquad\text{for all }K\geq K_{\varepsilon}.
0<vol⁡(C)=vol⁡(ΨhK​(C))≤vol⁡(Bε​(𝐳∗))=c2​d​ε2​d,0<\mathrm{vol}(C)=\mathrm{vol}(\Psi_{h}^{K}(C))\leq\mathrm{vol}(B_{\varepsilon}(\mathbf{z}^{\ast}))=c_{2d}\varepsilon^{2d},

C.2 Scope and limitations of the guarantee

Three limits are essential. Volume preservation does not bound rollout error; an invertible map can still stretch badly. It also does not preserve a Gaussian feature distribution, so encoder regularization remains relevant. Finally, latent Hamiltonian structure does not identify the true physical Hamiltonian or make the encoder injective on physical states. Distinct inputs already merged by the encoder cannot be recovered by an invertible predictor.

Remark 3 (Shape, not size).

A cloud that cannot be squeezed can still be sheared and stretched arbitrarily, so symplecticity places no bound on the growth of rollout error and does not make the rollout accurate. What it removes is the degenerate failure in its three forms: the rollout that shrinks in every direction at once, the one that settles onto an attractor and stops moving, and the one that merges states which started apart.

An unconstrained predictor FθF_{\theta} is under no such constraint, and the one-step objective pushes it the other way. Under any observation or latent noise, the minimiser of 𝔼​∥Fθ​(𝐳t)−𝐳t+1∥2\mathbb{E}\lVert F_{\theta}(\mathbf{z}_{t})-\mathbf{z}_{t+1}\rVert^{2} is the conditional mean 𝔼⁡[𝐳t+1∣𝐳t]\mathbb{E}[\mathbf{z}_{t+1}\mid\mathbf{z}_{t}], which averages away whatever the anchor leaves undetermined: it contracts volume, and it sends latents that differ only in the averaged-out directions to a common image. Volume loss and merging are therefore the trained optimum for such a map and iterating it drives the rollout onto exactly the attracting set the proposition forbids for Ψh\Psi_{h}. The proposition says the Hamiltonian rollout cannot enter that regime at any step size hh, any parameter ϕ\phi or any horizon.

Remark 4 (Regularization is not made redundant).

Proposition 1 states that the flow carries phase-space volume exactly and its does not state nothing about the shape of the distribution that volume belongs to. Due to this, regularization techniques are not made redundant. In this work, we leverage the SIGReg (Balestriero and LeCun, 2025) as regularizer, as it is the one used in LeWM and has shown state-of-the-art performance (Maes et al., 2026b). For the isotropic Gaussian 𝒩⁡(𝟎,𝐈)\mathcal{N}(\mathbf{0},\mathbf{I}) installed by SIGReg at the encoder to be invariant under the flow, the radius ∥(𝐪,𝐩)∥2\lVert(\mathbf{q},\mathbf{p})\rVert^{2} would itself have to be a conserved quantity of the generator, that is {∥(𝐪,𝐩)∥2,Hϕ}=0\{\lVert(\mathbf{q},\mathbf{p})\rVert^{2},H_{\phi}\}=0. This is a genuine restriction: it is a functional constraint on HϕH_{\phi}, not a consequence of symplecticity, and a general learned Hamiltonian, which is action-conditioned, and parameterised by free-form networks as we do here, has no reason to satisfy it.

The two mechanisms are therefore complementary rather than substitutes. SIGReg shapes the encoder distribution, during training and in expectation over batches, which is where a collapse of the representation would originate. Symplecticity carries the volume of whatever distribution it is handed, at every inference step and for every input, which is where an iterated map would otherwise contract or explode.

Remark 5 (Geometric prior, no physical modeling).

We are not modeling the physics of the observed system. HϕH_{\phi} is never fit to a known Hamiltonian and is not claimed to recover one: the true equations of motion enter only as evaluation targets for frozen read-out probes, never as a training signal. This is the point of departure from Hamiltonian neural networks (Greydanus et al., 2019) and SRNN (Chen et al., 2019), where the learned HϕH_{\phi} is a surrogate for the physical Hamiltonian and recovering it is the goal.

The prior models the latent representation space. Writing ℱ\mathcal{F} for the maps a JEPA predictor may realize, we restrict it to the symplectomorphisms reachable as one leapfrog step of a learned separable generator Hϕ​(𝐪t,𝐩t,𝐮t)H_{\phi}(\mathbf{q}_{t},\mathbf{p}_{t},\mathbf{u}_{t}); the content of that restriction is geometric (Proposition 1), and it governs the dynamics, or evolution, latent space. The canonical split is structural in the same sense: the encoder emits 𝐬t=(𝐪t,𝐩t)\mathbf{s}_{t}=(\mathbf{q}_{t},\mathbf{p}_{t}) with 𝐪t,𝐩t∈ℝd\mathbf{q}_{t},\mathbf{p}_{t}\in\mathbb{R}^{d} and the flow operates on that split by construction, but 𝐪t\mathbf{q}_{t} is not assumed to encode position, nor 𝐩t\mathbf{p}_{t} momentum, in any observation-space sense.

C.3 Why Hamiltonian and not Lagrangian

A separable Hamiltonian permits cheap explicit first-order updates on a paired state through leapfrog. This is the implementation reason for choosing it over a second-order Lagrangian formulation. The choice does not make the two physical formalisms inequivalent.

C.4 Design choices in the flow and the encoder

Actions enter the potential and stay constant during each integration step. The derivative stencil is causal for input anchors and centered only for training targets, which never feed the rollout. Defining momentum as the derivative of learned position fixes a useful coordinate convention, but it is not a general derivation of canonical physical momentum. Constant-mass systems can fit this convention; configuration-dependent mass can make the separable model misspecified. The learned scale and alternative SIGReg placements affect optimization without replacing the geometric guarantee.

Equations 9–12
Hϕ​(𝐪,𝐩,𝐮)=Kϕ1​(𝐩)+Vϕ2​(𝐪,𝐮),H_{\phi}(\mathbf{q},\mathbf{p},\mathbf{u})\;=\;K_{\phi_{1}}(\mathbf{p})\;+\;V_{\phi_{2}}(\mathbf{q},\mathbf{u}), (9)
Hϕ​(𝐳t+1,𝐮t+1)−Hϕ​(𝐳t,𝐮t)=Hϕ​(𝐳t+1,𝐮t+1)−Hϕ​(𝐳t+1,𝐮t)⏟Wt+Hϕ​(𝐳t+1,𝐮t)−Hϕ​(𝐳t,𝐮t)⏟rt,H_{\phi}(\mathbf{z}_{t+1},\mathbf{u}_{t+1})-H_{\phi}(\mathbf{z}_{t},\mathbf{u}_{t})\;=\;\underbrace{H_{\phi}(\mathbf{z}_{t+1},\mathbf{u}_{t+1})-H_{\phi}(\mathbf{z}_{t+1},\mathbf{u}_{t})}_{W_{t}}\;+\;\underbrace{H_{\phi}(\mathbf{z}_{t+1},\mathbf{u}_{t})-H_{\phi}(\mathbf{z}_{t},\mathbf{u}_{t})}_{r_{t}}, (10)
∇𝐩Hϕ(𝐪,𝐩,𝐮)=𝐒−1∇Kϕ(𝐒−1𝐩)=𝐩,\nabla_{\mathbf{p}}H_{\phi}(\mathbf{q},\mathbf{p},\mathbf{u})\;=\;\mathbf{S}^{-1}\nabla K_{\phi}\!\left(\mathbf{S}^{-1}\mathbf{p}\right)\;=\;\mathbf{p}, (11)
Kϕ​(𝐒−1​𝐩)=12​‖𝐩‖2+Cϕ,K_{\phi}\!\left(\mathbf{S}^{-1}\mathbf{p}\right)\;=\;\tfrac{1}{2}\|\mathbf{p}\|^{2}+C_{\phi}, (12)
𝒯={h,…,T−1−h}={h,…,A+K},|𝒯|=10,\mathcal{T}=\{h,\dots,T-1-h\}=\{h,\dots,A+K\},\qquad|\mathcal{T}|=10,
𝐙t=[(qt(i),pt(i))]i=1B∈ℝB×2​d,𝒮=1|𝒯|​∑t∈𝒯SIGReg⁡(𝐙t).\mathbf{Z}_{t}=\bigl[(q_{t}^{(i)},p_{t}^{(i)})\bigr]_{i=1}^{B}\in\mathbb{R}^{B\times 2d},\qquad\mathcal{S}=\frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}}\mathrm{SIGReg}(\mathbf{Z}_{t}).
𝐐t=[qt(i)]i=1B,𝐏t=[pt(i)]i=1B∈ℝB×d,𝒮=1|𝒯|​∑t∈𝒯[SIGReg⁡(𝐐t)+SIGReg⁡(𝐏t)].\mathbf{Q}_{t}=\bigl[q_{t}^{(i)}\bigr]_{i=1}^{B},\ \ \mathbf{P}_{t}=\bigl[p_{t}^{(i)}\bigr]_{i=1}^{B}\in\mathbb{R}^{B\times d},\qquad\mathcal{S}=\frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}}\Bigl[\mathrm{SIGReg}(\mathbf{Q}_{t})+\mathrm{SIGReg}(\mathbf{P}_{t})\Bigr].

Appendix D Description of the experimental set up

D.1 Methodology

Comparisons share data, clipping, optimizer and the total latent width, while LeHami splits that width into two halves. Checkpoints use validation data. This controls several confounds, but the derivative state and predictor both change together.

D.2 Implementation of LeHami

The implementation uses smooth MLP generator terms and fixed derivative stencils. The same causal derivative initializes training, evaluation and planning; centered targets provide lower-noise supervision. Once rollout begins, no new stencil or image supplies its state. Mixed-recording-rate training is described as an architectural possibility and explicitly was not tested.

xt−2x_{t-2}xt−1x_{t-1}xtx_{t}W=3W=3 frame window  3×64×643{\times}64{\times}64stem   Conv3×33{\times}3→323\!\to\!32 + BN + GELU + MaxPool run once per frame, shared across overlapping windows temporal mix   Conv1d over tt, 32→3232{\to}32kernel tied (a,b,a)(a,\,b,\,a)   at 32×3232{\times}32 spatial res.3×(32×32×32)3\times(32{\times}32{\times}32)trunk (unchanged)   →→→19232\!\to\!64\!\to\!128\!\to\!1923 conv blocks + AdaptiveAvgPool + Flatten32×32×3232{\times}32{\times}32projector MLP 192→2048→d192\to 2048\to d192192qt∈ℝdq_{t}\in\mathbb{R}^{d}pt=SG⁡(qt−3..t)p_{t}=\mathrm{SG}(q_{t-3..t})causalleapfrog (KDK)   H=K⁡(p/s)+V⁡(q,u)H=K(p/s)+V(q;u)pt+12=pt−Δ​t2​∇qV​(qt,u)qt+1=qt+Δ​t​∇pK​(pt+12/s)pt+1=pt+12−Δ​t2​∇qV​(qt+1,u)\begin{aligned} p_{t+\frac{1}{2}}&=p_{t}-\tfrac{\Delta t}{2}\,\nabla_{q}V(q_{t};u)\\[-1.0pt] q_{t+1}&=q_{t}+\Delta t\,\nabla_{p}K(p_{t+\frac{1}{2}}/s)\\[-1.0pt] p_{t+1}&=p_{t+\frac{1}{2}}-\tfrac{\Delta t}{2}\,\nabla_{q}V(q_{t+1};u)\end{aligned}s=eσs=e^{\sigma} learnable per-dimension momentum scale (qt+1,pt+1)(q_{t+1},p_{t+1}) — one dataset frame ahead
Figure 6: The temporal ConvNet encoder inside the LeHami framework, drawn at W=3W=3. A window of frames goes through a stem applied once per frame, then a single Conv1d over time with a symmetric tied kernel (a,b,a)(a,b,a), then the baseline’s spatial trunk and projector, unchanged, to give 𝐪t\mathbf{q}_{t}. The same encoder is used by the W=3W{=}3 LeWM baselines; the main LeHami model uses W=1W{=}1, i.e. no temporal mix. The rollout is initialised with the causal Savitzky–Golay momentum 𝐩t\mathbf{p}_{t}, computed from the current and three previous positions only, and advanced one frame at a time by the leapfrog step (orange) of Hϕ=Kϕ​(𝐩/s)+Vϕ​(𝐪,𝐮)H_{\phi}=K_{\phi}(\mathbf{p}/s)+V_{\phi}(\mathbf{q},\mathbf{u}), where the learnable per-dimension scale ss (blue) rescales the momentum and therefore appears only in the position update.

D.3 Datasets and their architectures

D.3.1 Conservative pendulum.

The conservative dataset varies length and includes oscillating and rotating trajectories. Only the stated single-frame-rate slice is used in the main experiment. Rendered rod length is visible from an image, making it a useful system-parameter control.

Equation 13
θ¨=−gL​sin⁡θ,\ddot{\theta}=-\frac{g}{L}\sin\theta, (13)
Refer to caption
Figure 7: Frames from the three rod lengths of (L∈{0.7,1.0,1.4}​mL\in\{0.7,1.0,1.4\}\,\mathrm{m}), rendered to scale at 64×6464\times 64. The rod length is therefore readable from a single frame, which is what makes it a positive control for the probes rather than a prediction target.

D.3.2 Action as a force: actuated pendulum.

The driven dataset varies length and mass and holds torques over three frames. Drive changes energy while retaining the volume-preserving dynamics used by the model. This is different from frictional energy loss.

Equation 14
θ¨=−gL​sin⁡θ+um​L2,\ddot{\theta}=-\frac{g}{L}\sin\theta+\frac{u}{mL^{2}}, (14)

D.3.3 The data split protocol

Whole episodes are split into training, validation and test, stratified by system-parameter combinations. Overlapping clips are drawn only after that split, preventing shared episode frames across partitions.

Table 3: The two environments and their episode-level splits. Both use the same protocol, described in Appendix D.3.
conservative pendulum actuated pendulum
episodes used 300 900
frames per episode 80 80
Δ​t\Delta t 0.100.10\,s 0.100.10\,s
split, in episodes (train / val / test):
episodes 𝟐𝟏𝟎/𝟒𝟓/𝟒𝟓\mathbf{210/45/45} 𝟔𝟑𝟎/𝟏𝟑𝟓/𝟏𝟑𝟓\mathbf{630/135/135}
design cells 3 rod lengths 9 (3​L×3​m3L\times 3m)
episodes per cell 100 100
train episodes per cell 70 70

D.4 Evaluation

The evaluation uses physical readout probes and executed simulator trajectories, with separate metrics for representation quality and control.

D.4.1 The probes

Angle is read as sine and cosine and converted back with atan2, avoiding a spurious discontinuity at a full turn. Probes fitted after one step help compare models with different access to motion history. Linear and nonlinear probes reveal different degrees of decodability.

D.4.2 Planning

Only the driven pendulum has actions to optimize. The controller uses encoded start context and a supplied goal window; physical state is reserved for scoring.

Goal construction.

Goals are actual later states from held-out episodes, making them reachable under some action sequence. This controls impossible targets, but the benchmark still supplies an informative target observation and motion window.

Latent states, and the frames they are read from.

Start and goal latents use the frames each architecture needs. LeHami’s goal includes history-derived momentum, while a single-frame baseline goal cannot directly specify velocity. Temporal-window variants address that difference.

Optimisation.

CEM ranks action candidates by terminal latent distance. Replanning shrinks the remaining horizon toward a fixed goal time and uses newly observed context. This avoids continually moving the deadline into the future.

No leakage.

Intermediate imagined states are self-fed. The source states that future episode states and recorded future actions do not initialize or rank candidate plans. Replanning contexts contain only observations already available to the controller.

Scoring success.

Success is measured at the target time using separate angle and signed-velocity tolerances. Angle differences are wrapped on the circle. Temporal encoder lag is accounted for when building online contexts. Passing near the goal at any earlier time is a different and easier criterion.

Equation 15
Δ​θ=wrap⁡(θT−θg),wrap⁡(a)=((a+π)mod2​π)−π,\Delta\theta=\operatorname{wrap}(\theta_{T}-\theta_{g}),\qquad\operatorname{wrap}(a)=\bigl((a+\pi)\bmod 2\pi\bigr)-\pi, (15)

Settings.

The protocol fixes a thirty-frame goal horizon, action bounds and a CEM budget, then sweeps replanning frequency. Results combine model and planner seeds on the same fifty goals. FLOP limits constrain which feedback rates can be compared.

Appendix E Supplementary experimental results

The supplementary experiments vary encoder design, regularization, prediction horizon and planning feedback. They explain both the observed improvements and the remaining failure cases.

E.1 LeHami replaces manual design choices with structural priors and lower compute

A time-symmetric encoder can recover speed while discarding its sign: reversing its frame window leaves the feature unchanged. An asymmetric derivative or untied temporal weights preserve that distinction. LeHami improves compute-to-error performance in the reported conservative-pendulum study, but that training-compute comparison uses one seed per model.

Refer to caption
Figure 8: The LeHami embeddings on the actuated pendulum are canonical. R2R^{2} of the pendulum angle θ\theta (left) and angular velocity ω\omega (right) decoded from the position half qq (solid) and momentum half pp (hatched) of the frozen encoder output. Probes are 25-NN under five episode-grouped folds fitted on embedded anchor frames (K=0K=0) pooled across test episodes. R2R^{2} of θ\theta is evaluated over (sin⁡θ,cos⁡θ)(\sin\theta,\cos\theta). Bars indicate the mean over three training seeds; error bars denote standard deviation. LeWM has no partitioned momentum block and is reported as a single joint latent zz.
Table 4: MAE at a fixed training budget for the conservative pendulum. Test error at the checkpoint with the lowest validation prediction loss within each compute budget (epoch in parentheses), decoded through a frozen kk-NN probe at k=20k=20 to ensure a shared scale. Checkpoints were saved every 10 epochs; evaluated on one seed per model.
Metric Model 0.750.75 PFLOP 1.51.5 PFLOP 33 PFLOP 66 PFLOP
MAE θ\theta [rad] LeHami W=1W{=}1 0.283\mathbf{0.283} (ep 9) 0.207\mathbf{0.207} (ep 19) 0.252\mathbf{0.252} (ep 39) 0.222\mathbf{0.222} (ep 79)
LeWM W=1W{=}1 0.590 (ep 9) 0.871 (ep 19) 0.487 (ep 39) 0.321 (ep 79)
MAE ω\omega [rad/s] LeHami W=1W{=}1 1.002\mathbf{1.002} (ep 9) 0.929\mathbf{0.929} (ep 19) 0.934\mathbf{0.934} (ep 39) 0.902\mathbf{0.902} (ep 79)
LeWM W=1W{=}1 1.625 (ep 9) 2.146 (ep 19) 1.620 (ep 39) 1.270 (ep 79)
Table 5: Parameter counts of the models trained on the conservative pendulum environment. The values are shown in millions, measured from the checkpoints. The predictor is where the two families differ: LeHami’s separable Hamiltonian flow (K⁡(p)+V⁡(q)K(p)+V(q) MLPs, hidden_dim =1024=1024) against LeWM’s 6-layer autoregressive transformer. pred. proj. is LeWM’s predictor projection head, which LeHami has no counterpart for; the encoder and the latent projector are matched across families, and the W=3W{=}3 encoder adds only ∼\sim3k parameters over W=1W{=}1.
Model Total Encoder Predictor Projector Pred. proj.
LeHami W=1W{=}1 3.03 0.315 2.234 0.465 —
LeWM W=1W{=}1 4.44 0.315 3.310 0.530 0.268
LeWM W=3W{=}3 4.45 0.318 3.310 0.530 0.268

E.2 Conservative pendulum

Latent plots illustrate structured orbits, while probes test whether physical information remains decodable. Energy-like organization and unit Jacobian determinant do not ensure accurate long-rollout coordinates: the model’s momentum features drift from their encoded targets. The full three-state context map is used to assess LeWM volume, since its newest-output block alone is not a complete state-transition map. Some frozen-encoder controls use a separate single-length dataset and one seed.

Refer to caption
Figure 9: Conservative pendulum: rollout latents projected onto pairs of principal components. Each panel is one pair of the leading PCs of the encoder’s (𝐪,𝐩)(\mathbf{q},\mathbf{p}) over the test episodes; colour is the episode. The learned HϕH_{\phi} is conserved along each rollout, so an episode traces a closed curve in every projection rather than drifting across them.
Refer to caption
Figure 10: Latent representation space projected into the first two principal components (PCs). The embeddings correspond to LeHami (left) and unstuctured LeWM (right), shown as a function (z-axis) of the Hamiltonian learned (left) and the physical energy corresponding to the episodes. The embeddings of four episodes are shown in colors. The planes with the same color correspond to the learned Hamiltonian and energy, respectively. While the rollout episode’s embeddings in LeHami are restrictive in each of the planes corresponding to the (constant) respective learned Hamiltonian, in LeWM, the rollout episode’s representations move across planes of energy, mixing up.
Refer to caption
Figure 11: The learned energy landscape against the physical one. (left) level sets of the true energy of the conservative pendulum; (right) level sets of the learned HϕH_{\phi} over the same 6060 test rollouts, each started from its own anchor. Distinct energies give disjoint closed curves in both, so the learned generator stratifies the latent space the way the physical energy stratifies phase space.
Table 6: Where the LeHami’s two blocks lose the reference, conservative pendulum. R2R^{2} of the predicted 𝐪k\mathbf{q}_{k} and 𝐩k\mathbf{p}_{k} against the encoded ground-truth 𝐪\mathbf{q} and 𝐩\mathbf{p} at the same frame, test split. These are latent-space quantities in the model’s own coordinates, not physical ones: they measure how far the flow drifts from the encoder trajectory it was trained to reproduce, and being self-referential they cannot by themselves establish that a physical quantity survives. What survives decoding is in Table 7..
rollout step kk 0 1 5 10 20 25 50
Rq2R^{2}_{q} (position block) 1.0001.000 0.9970.997 0.9850.985 0.9500.950 0.9290.929 0.6460.646 0.3730.373
Rp2R^{2}_{p} (momentum block) 1.0001.000 0.9350.935 0.8670.867 0.6130.613 0.5860.586 −0.324-0.324 −1.165-1.165
Table 7: Performance in physics prediction of the conservative pendulum. Wrapped MAE of angle θ\theta and ω\omega predicted by the kk-NN probe (with k=25k=25) fitted on each model’s own K=1K=1 output and applied frozen along the rollout, averaged over the rollout band. Ratio is LeWM ÷\div LeHami, so >1>1 favours the LeHami.
MAE θ\theta [rad, wrapped] MAE ω\omega [rad/s]
band LeHami LeWM ratio LeHami LeWM ratio
k=1k=1–1010 0.107 0.100 0.93 0.364 1.020 2.80
k=11k=11–2020 0.225 0.287 1.27 0.669 1.169 1.75
k=21k=21–7070 0.596 0.981 1.65 1.534 2.331 1.52
anchor (K=0K=0), R2​(ω)R^{2}(\omega) from the frozen encoder features:  LeHami +0.975+0.975  LeWM −0.667-0.667
same anchor, LeHami read by the linear tier:  ridge +0.448+0.448, against the 25-NN’s +0.975+0.975 above
Refer to caption
Figure 12: DINO-LeHami vs. DINO-WM on the conservative pendulum. Both models use a frozen DINOv2 encoder followed by a learned projector; Wrapped MAE of θ\theta (left) and ω\omega (right) along the test rollout (KK steps, Δ​t=0.05\Delta t=0.05 s), read by a kk-NN probe (k=25k=25) fitted at K=1K=1 and kept frozen. One training seed per model, both at epoch 6060.
Refer to caption
Figure 13: Log-volume factor of the one-step latent map on the conservative pendulum. One Jacobian per test episode at the anchor (4545 states; bars mark the mean). Left: the full map, n=64n=64 for LeHami on (𝐪,𝐩)(\mathbf{q},\mathbf{p}) and n=192n=192 for LeWM on its three-embedding context. Right: the newest block ∂𝐳t+1/∂𝐳t\partial\mathbf{z}_{t+1}/\partial\mathbf{z}_{t}, n=64n=64 for both. The dashed line is the true flow. One training seed per model.

E.3 Actuated pendulum

Driving changes the learned energy along a trajectory without removing symplecticity. Joint Gaussian regularization can hide a nearly inactive momentum block by inflating position variance; placement therefore matters. A symmetric temporal baseline confuses opposite velocity signs, whereas untied weights correct that failure. Both model families retain mass and length, so those system parameters do not explain the control gap.

Refer to caption
Figure 14: The learned energy landscape under actuation. The same projection as Fig. 11, for the actuated pendulum. The torque does work, so HϕH_{\phi} is not constant along a rollout and the trajectories cross the level sets instead of nesting into disjoint shells. Symplecticity is unaffected: it constrains the map, not the value of HϕH_{\phi}.
Table 8: Within- versus between-episode variance of the encoder’s latent coordinates, by SIGReg placement. For each latent dimension, VwithinV_{\text{within}} is the mean squared deviation of an episode’s trajectory about its own mean and VbetweenV_{\text{between}} is the variance of those per-episode means; the column reports Vwithin/(Vwithin+Vbetween)V_{\text{within}}/(V_{\text{within}}+V_{\text{between}}), averaged over dimensions. A value near zero means the latent is an episode identifier rather than a state. Measured on the encoder’s own latents (gt_q, gt_p) over the test episodes at a common horizon K=0​…​74K=0\ldots 74; mean ±\pm standard deviation over three training seeds. Bold marks the largest within-episode share per window. The unstructured baseline attains 42.8%42.8\% on this measure but cannot be tabulated here, as its evaluation path stores latents only at K=0K=0.
window SIGReg within-episode % std⁡(q)\mathrm{std}(q) std⁡(p)\mathrm{std}(p) std⁡(p)/std⁡(q)\mathrm{std}(p)/\mathrm{std}(q)
W=1W=1 joint 0.01±0.000.01\pm 0.00 1.356 0.006 0.0043
q 15.32±0.2915.32\pm 0.29 0.980 0.101 0.1031
split 48.89±16.39\mathbf{48.89\pm 16.39} 1.174 0.236 0.2014
W=3W=3 joint 4.90±0.304.90\pm 0.30 1.405 0.051 0.0362
q 8.87±0.548.87\pm 0.54 0.979 0.051 0.0519
split 47.06±2.56\mathbf{47.06\pm 2.56} 1.124 0.201 0.1784
Equation 16
b+2​a​cos⁡(ω​d​t)b+2​a​(sin⁡θ,cos⁡θ),\frac{b+2a\cos(\omega\,dt)}{b+2a}\,(\sin\theta,\ \cos\theta), (16)
Table 9: The comparison of the R2R^{2} due to the time window in the encoder. They correspond to the kk-NN (k=25k{=}25) probe applied on encoded anchor latents over held-out episodes, across 3 seeds. The nn refers to the history size of the predictor, which is the same in both. Note that W=3W=3 refers here to the tied variant, which exacerbates the sign-blindness gap.
Model nn R2​(ω)R^{2}(\omega) R2​(|ω|)R^{2}(|{}\omega|{}) gap
LeWM W=1W{=}1 3 −0.048-0.048 +0.032+0.032 +0.080+0.080
LeWM W=3W{=}3 3 −0.115-0.115 +0.921+0.921 +1.036+1.036
Refer to caption
Figure 15: Performance in physics predictions in rollout of the actuated pendulum. Wrapped MAE of θ\theta (left) and ω\omega (right) along the test rollout, 25-NN probe fitted at K=1K=1 and applied frozen. Same encoder, same schedule, same regularizer weight. Shaded area corresponds to the standard deviation across the three architecture seeds.
Refer to caption
Figure 16: The 5050 planning goals. Left: the goal velocity ω∗\omega^{\color[rgb]{0,0.5,0.5}*}; dashed lines mark ±τω/2\pm\tau_{\omega}/2, beyond which a sign error alone fails the velocity clause. 92%92\% of the goals lie outside them. Right: the goal angle θ∗\theta^{\color[rgb]{0,0.5,0.5}*}, wrapped; the green lines mark upright, which no goal is within τθ\tau_{\theta} of. Goals are recorded states of held-out test episodes at K=30K=30 frames after the start, shared identically by every model, rung and seed.

Planning compute.

Compute savings come mainly from cheaper predictor rollouts. LeHami improves velocity direction relative to passive coasting and symmetric baselines, but the untied baseline performs better overall. Conditioning joint success on reaching the angle shows that position control is LeHami’s main bottleneck here. Recoverable mass and length do not imply every latent coordinate remains linearly calibrated.

Table 10: Velocity clause decomposition into magnitude and directional sign components. Evaluated under the same computational budget (≤15\leq 15 TFLOP/episode) and protocol as Table 2.
Clause Model Success TFLOP Replanning Interval (RR)
Velocity, magnitude LeWM W=1W{=}1 0.251±0.0510.251\pm 0.051 8.78 R=15R=15, 2 solves
LeWM W=3W{=}3 (tied) 0.638±0.0080.638\pm 0.008 13.65 R=9R=9, 4 solves
LeWM W=3W{=}3 (untied) 0.813±0.052\mathbf{0.813\pm 0.052} 13.65 R=9R=9, 4 solves
LeHami 0.547±0.0220.547\pm 0.022 10.98 R=6R=6, 5 solves
u=0u=0 coast 0.300 — —
Velocity, sign LeWM W=1W{=}1 0.656±0.0170.656\pm 0.017 5.85 R=30R=30, 1 solve
LeWM W=3W{=}3 (tied) 0.649±0.0220.649\pm 0.022 13.65 R=9R=9, 4 solves
LeWM W=3W{=}3 (untied) 0.962±0.017\mathbf{0.962\pm 0.017} 13.65 R=9R=9, 4 solves
LeHami 0.800±0.0340.800\pm 0.034 8.05 R=9R=9, 4 solves
u=0u=0 coast 0.680 — —
Table 11: Performance in prediction of rod’s length and mass in the actuated pendulum. Both episode-constant parameters are recovered from the latent by probes fitted at anchor (K=0K=0) on training episodes and applied unchanged along each model’s own rollout over 135135 held-out episodes produced by LeHami and LeWM. R2R^{2} is reported at the anchor and at the end of the horizon (K=76K=76). Three seeds for LeHami (min–max in brackets) for both ridge and 2525-NN probes. Both models carry both parameters. While the kk-NN (k=25) probe is exact at every KK rollout, it is observed some drift for the ridge, affecting more LeHami. Note that The R2R^{2} columns therefore measure how far the frozen readout drifts, not whether the parameter is still identifiable.
rod length LL bob mass mm
probe model K=0K=0 K=76K=76 K=0K=0 K=76K=76
ridge LeHami 0.999 0.900 [0.871, 0.951] 0.999 0.977 [0.958, 0.993]
LeWM 0.999 0.983 [0.976, 0.990] 0.999 0.993
2525-NN LeHami 1.000 0.997 [0.992, 1.000] 1.000 1.000
LeWM 1.000 1.000 1.000 1.000
Concept demonstration

Open image ↗