Literature
Display
← All papers

Critique of World Model

Eric Xing · Mingkai Deng · Jinyu Hou

· arXiv (preprint) ↗

World Models8 min text · CondensedOriginal paper ↗
Contents

At a glance

This essay argues that world models should support action-conditioned simulation for reasoning, planning, and agent learning. It organizes the design problem around data, representation, architecture, training objectives, and usage.

Its proposed GLP architecture combines discrete concepts, continuous perceptual features, and observation reconstruction. PAN sketches how those components might support hierarchical agents. This paper presents a position and architectural proposal, not a controlled benchmark demonstrating general intelligence. Some stated theoretical guarantees require stronger assumptions, identified below.

1 Introduction

The motivating idea is to compare imagined futures before acting. A useful world model should predict consequences across physical, social, and abstract situations, rather than merely generate plausible imagery. The authors advocate multimodal experience, mixed representations, hierarchical prediction, observation-grounded training, and policy learning from simulated experience.

This edition follows arXiv v5, revised June 2026; its publication date records the first preprint posting. Detailed PAN experiments belong to the separately cited companion paper. [66]

Equation 1
s′∼p⁡(s′|s,a)s^{\prime}\sim p(s^{\prime}|s,a) (1)

2 World Model and Agent Decision-Making

The formal setup pairs an agent’s action policy with an environment transition model and a goal-dependent reward. A learned world model supplies a usable surrogate when real interaction or exact dynamics are unavailable.

2.1 Agent-Environment Model and Optimal Agent

The paper writes trajectory probability as a product of action and environment-transition probabilities. Expected discounted reward defines a value function and motivates selecting actions by predicted future return. This idealized presentation assumes access to a sufficient state representation and a well-defined discounted objective.

Refer to caption
Figure 1: A possible definition of an optimal agent
Equations 2–5
pμπ​(at,st+1,…,sT∣st)=∏k=tT−1pπ​(ak∣sk)⏟ agent ​pμ​(sk+1∣sk,ak)⏟ universe\displaystyle p^{\pi}_{\mu}(a_{t},s_{t+1},\dots,s_{T}\mid s_{t})=\prod_{k=t}^{T-1}{\underbrace{\textstyle p_{\pi}(a_{k}\mid s_{k})}_{\text{ agent }}}\ {\underbrace{\textstyle p_{\mu}(s_{k+1}\mid s_{k},a_{k})}_{\text{ universe }}} (2)
Vπ,μg​(st)\displaystyle V_{\pi,\mu}^{g}(s_{t}) :=𝔼π,μ[∑k=t∞γkr(g,sk)|st]\displaystyle\vcentcolon=\mathbb{E}_{\pi,\mu}\left[\sum_{k=t}^{\infty}\gamma_{k}r(g,s_{k})\mathrel{\bigg|}s_{t}\right]
=limT→∞∑(at,st+1,…,sT)∑k=tTγk​r​(g,sk)⏟goal​pμπ​(at,st+1,…,sT∣st)⏟trajectory\displaystyle=\lim_{T\rightarrow\infty}\sum_{(a_{t},s_{t+1},\dots,s_{T})}{\underbrace{\textstyle\sum_{k=t}^{T}\gamma_{k}r(g,s_{k})}_{\text{goal}}}\ {\underbrace{\textstyle p^{\pi}_{\mu}(a_{t},s_{t+1},\dots,s_{T}\mid s_{t})}_{\text{trajectory}}} (3)
πμ∗:=arg​maxπ⁡Vπ,μg\pi^{*}_{\mu}:=\argmax_{\pi}V^{g}_{\pi,\mu} (4)
πμ∗(st)=arg​maxat:T−1⏟possible actions∑st+1:T(∑k=tT−1γk​r​(g,sk)+γT​Vπ,μg​(sT)⏟goal progress)∏i=tT−1pμ​(si+1|si,ai)⏟universe response\pi^{*}_{\mu}(s_{t})={\underbrace{\argmax_{a_{t:T-1}}}_{\text{possible actions}}}\ \sum_{s_{t+1:T}}\Bigg({\underbrace{\sum_{k=t}^{T-1}\gamma_{k}r(g,s_{k})+\gamma_{T}V_{\pi,\mu}^{g}(s_{T})}_{\text{goal progress}}}\Bigg)\prod_{i=t}^{T-1}\ {\underbrace{p_{\mu}(s_{i+1}|s_{i},a_{i})}_{\text{universe response}}}\ (5)

2.2 World Model and Simulative Reasoning

A practical agent encodes observations into an internal belief state, proposes actions, and rolls the world model forward. A critic evaluates those simulated futures. The authors emphasize persistent memory, multiple time scales, counterfactuals, and domains beyond physical motion.

These are desired capabilities of a general system. A learned simulator’s predictions and policy quality remain limited by its data, state representation, model error, and planning procedure.

Refer to caption
Figure 2: An agent in real world where groundtruth world state and universe are unavailable to experience or experiment, so world model is crucial for simulation.
Equation 6
πf∗(s^t)=arg​maxa′t:T′−1⏟possible actions∑s^t+1:T′(∑k=tT′−1γk​r​(g,s^k)+γT′​Vπ,fg​(s^T′)⏟goal progress)∏i=tT′−1pf​(s^i+1|s^i,ai′)⏟   simulation with  world model  \pi^{*}_{f}(\hat{s}_{t})={\underbrace{\argmax_{a^{\prime}_{t:T^{\prime}-1}}}_{\text{possible actions}}}\ \sum_{\hat{s}_{t+1:T^{\prime}}}\Bigg({\underbrace{\sum_{k=t}^{T^{\prime}-1}\gamma_{k}r(g,\hat{s}_{k})+\gamma_{T^{\prime}}V_{\pi,f}^{g}(\hat{s}_{T^{\prime}})}_{\text{goal progress}}}\Bigg)\prod_{i=t}^{T^{\prime}-1}\ {\underbrace{p_{f}(\hat{s}_{i+1}|\hat{s}_{i},a^{\prime}_{i})}_{{\scriptsize\shortstack{simulation with\\ world model}}}} (6)

3 The World Model Landscape

The survey groups contemporary systems by their main demonstrated capabilities. Its assessments reflect the essay’s cited versions and the authors’ preferred definition of a decision-oriented world model.

Gaming World Models

Game simulators produce action-conditioned visual trajectories, but the authors question their action diversity, interacting-agent support, and long-term consistency. These are scope limitations of the surveyed demonstrations, not a shared impossibility result.

3D Scene World Models

Scene generators create explorable geometry. The essay distinguishes generating a static environment from learning its interactive dynamics, especially when exported geometry relies on a separate programmed physics engine.

Physical World Models

Driving and robotics world models learn useful physical regularities in specific domains. The critique asks whether their sensor interfaces, action spaces, and training distributions can support broader social or multi-agent situations.

Video Generation Models

Prompted video generation can depict plausible events without providing persistent state or explicit alternative-action control. The authors therefore separate visual synthesis from the stronger simulation interface needed for evaluating decisions.

Joint Embedding Predictive Models

JEPA-style systems predict latent representations instead of reconstructing pixels. The authors recognize this efficiency and abstraction strategy while questioning generalization to more diverse, extended tasks. V-JEPA 2 includes real-robot planning demonstrations; the open question concerns breadth, rather than an absence of practical evidence. [4]

4 Critique of World Modeling

The critique examines five design preferences: sensory-first data, continuous embeddings, decoder-free latent prediction, latent-space losses, and online model-predictive control. It argues for alternatives or combinations according to the desired scope of the system.

Original figure
Figure 3: A systematic technical proposal that results from common wisdom on world modeling, most prominently instantiated by the joint embedding predictive architecture (JEPA).

4.1 Data: Information Density, Not Just Volume

Raw byte count does not measure useful information. Language compactly records concepts, explanations, and social knowledge; sensory streams provide physical detail and grounding. The authors favor learning from their complementary roles instead of selecting one modality by volume alone.

4.2 Representation: Continuous? Discrete? Or Both?

The proposed representation combines discrete concepts for compositional reasoning with continuous features for fine perceptual variation. “Stateful” here means a persistent, identifiable estimate useful for predicting relevant dynamics. Discreteness alone does not establish that property, and a compressed continuous representation can also preserve sufficient task information.

Theorem 1 gives a bounded-input, finite-resolution encoding argument: sufficiently different real-valued inputs can receive distinct finite token sequences. Increasing sequence length avoids the rapidly growing vocabulary in the particular fixed-length construction. This establishes representational capacity, not semantic understanding, exact lossless encoding, learnability, or a general compute-efficiency advantage. [55]

Original figure
Figure 4: Vocabulary-based tokens is an effective way to categorize perceptual inputs into discrete concepts for reasoning (left). We may scale up or scale out discrete code to deal with increasing data complexity (right). Thm.1 shows either is effective, but scaling out is more efficient.

4.3 Architecture: Pure Abstraction, or Autoregressive Generation

The architectural proposal combines latent reasoning with a decoder that lets predictions be checked against observations.

4.3.1 Two Competing Approaches

Observation-generating systems explicitly reconstruct outputs; JEPA-style systems predict a target embedding. Repeated latent prediction still rolls a state forward recursively, even without a probabilistic observation decoder. The authors distinguish this functional simulation from sampling uncertain outcomes.

Equations
s^=h⁡(o),s^′=f⁡(s^,a),\hat{s}=h(o),\quad\hat{s}^{\prime}=f(\hat{s},a),
pf​(s^′|s^,a)=δ⁡(s^′−f⁡(s^,a)),p_{f}(\hat{s}^{\prime}|\hat{s},a)=\delta(\hat{s}^{\prime}-f(\hat{s},a)),
Original figure
Figure 5: Comparison of the joint embedding predictive architecture (JEPA, left) and the proposed Generative Latent Prediction (GLP) architecture (right). JEPA operates in open loop, predicting the next latent state without reconstructing observations. GLP closes the loop through a decoder that validates predictions against observable data, while incorporating both an enhanced LLM backbone for discrete reasoning and a diffusion-based predictor for continuous dynamics.

4.3.2 Tradeoffs for General-Purpose Modeling

Abstraction can discard information needed by an unforeseen task, while reconstruction can spend capacity on irrelevant detail. The authors prefer reconstruction as a diagnostic and training signal for broadly useful representations. The argument is a design trade-off: lossy encoders can retain sufficient dynamics, and reconstruction alone does not guarantee that every important fact or rare event is modeled.

4.3.3 Generative Latent Prediction as a Synthesis

GLP places continuous generative prediction at a perceptual level, learned discrete tokens at an intermediate level, and language-based reasoning above them. An enhanced LLM and diffusion predictor exchange information within an encoder–decoder system. Claimed improvements in long-horizon reasoning and stability are architectural motivations requiring empirical evaluation.

4.4 Objective: Learning in Data Space, or Latent Space?

The unregularized latent loss admits a constant encoder and constant predictor with zero error: Proposition 1 establishes this collapsed optimum. It does not show that practical JEPA methods with additional training mechanisms necessarily collapse.

Proposition 2 claims reconstruction excludes such optima, but its proof requires decoder access to the target observations and independently distinguishable inputs. Distinct stochastic outcomes for the same input can still have an optimal constant deterministic prediction.

Reading note: Theorem 2 claims latent error is at most observation error plus round-trip error. This does not follow from its stated assumptions. For scalar h(o)=2o, g(s)=s/2, f=0, and next observation 1, round-trip error is zero, yet latent error is 2 and observation error is 1. The original formulas and diagram remain below; they should not be treated as an established universal ordering of the two objectives.

Equations 7–8
ℒlatent​(h,f)=𝔼(o,a,o′)∼𝒟​[‖f⁡(h⁡(o),a)−h⁡(o′)‖],\mathcal{L}_{\text{latent}}(h,f)=\mathbb{E}_{(o,a,o^{\prime})\sim\mathcal{D}}\left[\left\lVert f(h(o),a)-h(o^{\prime})\right\rVert\right], (7)
ℒgen​(h,f,g)=𝔼(o,a,o′)∼𝒟​[‖g∘f⁡(h⁡(o),a)−o′‖].\mathcal{L}_{\text{gen}}(h,f,g)=\mathbb{E}_{(o,a,o^{\prime})\sim\mathcal{D}}\left[\left\lVert g\circ f(h(o),a)-o^{\prime}\right\rVert\right]. (8)
Original figure
Figure 6: Comparison of latent-space reconstruction objectives (left) and generative data reconstruction objectives (right).
Equations
ℒlatent​(h,f)=𝔼(o,a,o′)∼𝒟​[‖f⁡(h⁡(o),a)−h⁡(o′)‖],\mathcal{L}_{\text{latent}}(h,f)=\mathbb{E}_{(o,a,o^{\prime})\sim\mathcal{D}}\left[\left\lVert f(h(o),a)-h(o^{\prime})\right\rVert\right],
ℒlatent​(h∗,f∗)=minh,f⁡ℒlatent​(h,f)\mathcal{L}_{\text{latent}}(h^{*},f^{*})=\min_{h,f}\mathcal{L}_{\text{latent}}(h,f)
ℒgen​(h,f,g)=𝔼(o,a,o′)∼𝒟​[‖g∘f⁡(h⁡(o),a)−o′‖],\mathcal{L}_{\text{gen}}(h,f,g)=\mathbb{E}_{(o,a,o^{\prime})\sim\mathcal{D}}\left[\left\lVert g\circ f(h(o),a)-o^{\prime}\right\rVert\right],
ℒgen​(h~,f~,g′)<ℒgen​(h′,f′,g′)\mathcal{L}_{\text{gen}}(\tilde{h},\tilde{f},g^{\prime})<\mathcal{L}_{\text{gen}}(h^{\prime},f^{\prime},g^{\prime})
ℒgen​(h~,f~,g′)<ℒgen​(h′,f′,g′)\mathcal{L}_{\text{gen}}(\tilde{h},\tilde{f},g^{\prime})<\mathcal{L}_{\text{gen}}(h^{\prime},f^{\prime},g^{\prime})
ℒlatent=‖f⁡(h⁡(o),a)−h⁡(o′)‖,ℒgen=‖g∘f⁡(h⁡(o),a)−o′‖.\mathcal{L}_{\text{latent}}=\left\lVert f(h(o),a)-h(o^{\prime})\right\rVert,\qquad\mathcal{L}_{\text{gen}}=\left\lVert g\circ f(h(o),a)-o^{\prime}\right\rVert.
s^∣o∼𝒩(h(o),I),o^∣s^∼𝒩(g(s^),I),and s^′∣s^,a∼𝒩(f(s^,a),I).\hat{s}\mid o\sim\mathcal{N}(h(o),I),\quad\hat{o}\mid\hat{s}\sim\mathcal{N}(g(\hat{s}),I),\quad\text{and }\hat{s}^{\prime}\mid\hat{s},a\sim\mathcal{N}(f(\hat{s},a),I).
ℒlatent≤ℒgen+ϵ,\mathcal{L}_{\text{latent}}\leq\mathcal{L}_{\text{gen}}+\epsilon,
Original figure
Figure 7: As Thm. 2 shows, the energy-based latent reconstruction loss (ℒlatent\mathcal{L}_{\text{latent}}) is upper bounded by the generative data reconstruction loss (ℒgen\mathcal{L}_{\text{gen}}) plus a small encoder-decoder reconstruction error (ϵ\epsilon). ϵ\epsilon is small in practice, meaning ℒlatent≤ℒgen\mathcal{L}_{\text{latent}}\leq\mathcal{L}_{\text{gen}} usually holds. Minimizing ℒlatent\mathcal{L}_{\text{latent}}, therefore, does not guarantee consistency with observed data, which is required for minimizing ℒgen\mathcal{L}_{\text{gen}}.

4.5 Usage: MPC or RL?

MPC repeatedly simulates candidate action sequences at decision time. Learning a policy from model-generated rollouts shifts some computation into training and can make later action selection faster. Both approaches still depend on model accuracy and can be combined with planning, value functions, or reactive control.

The authors favor using world models for learning as well as inference. Their discussion does not establish that MPC is generally limited to short-sighted behavior or that model-based RL automatically avoids unsafe or inaccurate imagined experience.

Equation
(at∗,…,aT−1∗)=arg​minat,…,aT−1∑k=tT−1C(g,f(s^k,ak)).(a_{t}^{*},\dots,a_{T-1}^{*})=\argmin_{a_{t},\dots,a_{T-1}}\sum_{k=t}^{T-1}C(g,f(\hat{s}_{k},a_{k})).
Original figure
Figure 8: Comparison of model-predictive control (left) and reinforcement learning from world-model-simulated experience (right) as approaches to using the world model for decision-making.
Equation 9
πf∗​(s^t)\displaystyle\pi^{*}_{f}(\hat{s}_{t}) =arg​maxπ𝔼π,f[∑k=t∞γkr(g,s^k)|s^t]\displaystyle=\argmax_{\pi}\mathbb{E}_{\pi,f}\left[\sum_{k=t}^{\infty}\gamma_{k}r(g,\hat{s}_{k})\mathrel{\bigg|}\hat{s}_{t}\right] (9)
=arg​maxπlimT→∞∑(at,s^t+1,…,s^T)∑k=tTγkr(g,s^k)∏i=tTpπ​(ai∣s^i)⏟select actionpf​(s^i+1∣s^i,ai).⏟   simulation with  world model  \displaystyle=\argmax_{\pi}\lim_{T\to\infty}\sum_{(a_{t},\hat{s}_{t+1},\dots,\hat{s}_{T})}\sum_{k=t}^{T}\gamma_{k}r(g,\hat{s}_{k})\ \prod_{i=t}^{T}{\underbrace{p_{\pi}(a_{i}\mid\hat{s}_{i})}_{\scriptsize\shortstack{select action}}}\ {\underbrace{p_{f}(\hat{s}_{i+1}\mid\hat{s}_{i},a_{i}).}_{\scriptsize\shortstack{simulation with \\ world model}}}

5 The PAN World Model

PAN instantiates the proposed principles: multimodal experience, persistent mixed representations, hierarchical GLP prediction, observation-based supervision, and simulated experience for agents. Figure 9 summarizes the intended information flow.

Refer to caption
Figure 9: Illustration of the generative latent prediction (GLP) architecture of PAN-World, our proposed framework for general-purpose world modeling. At each time point, PAN-World estimates the world state s^\hat{s} based on the current sensory observation oo, predicts the future world states s^′\hat{s}^{\prime} based on proposed actions aa for agentic reasoning, and reconstructs the future observations o^′\hat{o}^{\prime} for generative supervision and external usage.

5.1 A Motivating Usecase

A mountaineering expedition illustrates the desired range: route planning, gear use, movement, weather assessment, and social coordination operate at different time and spatial scales. This is a motivating scenario, not a reported deployment or task-success experiment.

5.2 The GLP Architecture for PAN World Model

A sensory encoder produces learned discrete tokens and continuous embeddings. An LLM handles conceptual sequences, a diffusion predictor handles detailed perceptual transitions, and a learned switch chooses how to combine them. A multimodal decoder reconstructs predicted observations.

The factorization describes hierarchical state inference and next-state prediction. Dynamic vocabulary growth and persistent memory are proposed mechanisms; this essay does not measure their reliability or establish general-purpose state sufficiency.

Equation
pPAN​(o′∣o,a)\displaystyle p_{\text{PAN}}(o^{\prime}\mid o,a)
=\displaystyle= ∑s^,s^′ph​(s^∣o)⏟encoder​pf​(s^′∣s^,a)⏟world model​pg​(o′∣s^′)⏟decoder\displaystyle\sum_{\hat{s},\hat{s}^{\prime}}{\underbrace{p_{h}(\hat{s}\mid o)}_{\scriptsize\shortstack{encoder}}}\ {\underbrace{p_{f}(\hat{s}^{\prime}\mid\hat{s},a)}_{\scriptsize\shortstack{world model}}}\ {\underbrace{p_{g}(o^{\prime}\mid\hat{s}^{\prime})}_{\scriptsize\shortstack{decoder}}}
=\displaystyle= ∑s^,s^′∏i=1Nph​(s^i∣s^<i,o)⏟   hierarchical world  state inference  ​∏j=1N′pf​(s^j′∣s^<j′,s^,a)⏟   switch-based next-state  prediction  ​pg​(o′∣s^′)\displaystyle\sum_{\hat{s},\hat{s}^{\prime}}{\underbrace{\prod_{i=1}^{N}p_{h}(\hat{s}_{i}\mid\hat{s}_{<i},o)}_{\scriptsize\shortstack{hierarchical world \\ state inference}}}\ {\underbrace{\prod_{j=1}^{N^{\prime}}p_{f}(\hat{s}^{\prime}_{j}\mid\hat{s}^{\prime}_{<j},\hat{s},a)}_{\scriptsize\shortstack{switch-based next-state \\ prediction}}}\ p_{g}(o^{\prime}\mid\hat{s}^{\prime})

5.3 Training the PAN World Model

Modules are first pretrained on suitable modality-specific data, then aligned through multimodal post-training. The proposal uses text for broad conceptual knowledge and physical observations for embodied detail, aiming to combine knowledge without requiring complete demonstrations of every complex task. Its sample-efficiency advantages are expectations rather than quantified results here.

5.4 Towards Agentic Reasoning with PAN

A PAN agent would precompute and cache candidate futures, then consult them alongside current beliefs and goals. This separates some simulation effort from immediate action selection. The section outlines an agent design rather than a validated solution to open-ended planning.

Refer to caption
Figure 10: Illustration of our proposed simulative reasoning agent powered by the PAN World Model. Unlike traditional RL agents that rely on reactive policies, or model-precitive control (MPC) agents that expensively simulates futures at decision time, this agent leverages a cache of precomputed simulations generated by the PAN WM. During decision-making, the agent selects actions based on its current beliefs and expected outcomes, enabling a more efficient, flexible, and intentional form of planning, which, as we argue, is closer to the flexibility of human reasoning.

6 Conclusion

The essay’s main contribution is a synthesis of world-model design choices and a proposal for mixed, hierarchical, observation-grounded simulation. Wider generalization, multiple agents, long time scales, and learning from imagined experience remain research directions. Evaluating those ambitions requires evidence beyond the architectural sketch and the qualified theoretical arguments.

Appendix A Proof for Theorem 1

The proof quantizes each coordinate of a bounded input finely enough that equal codes imply a bounded reconstruction difference. It constructs either a large vocabulary with fixed-length codes or longer base-coded sequences with a fixed vocabulary.

This is a finite-precision existence argument. The printed proof has inconsistent norm, index, and vocabulary-size notation; its central quantization idea does not establish natural-language semantics or compare total storage bits and runtime.

Equations 10–12
|𝒱|M≥⌈T​D​K~​ϵ~−1⌉T​D|\mathcal{V}|^{M}\geq\lceil\sqrt{TD}\tilde{K}\tilde{\epsilon}^{-1}\rceil^{TD}
(d1​(ct),d2​(ct),…,dD​(ct)),(d_{1}(c_{t}),d_{2}(c_{t}),\dots,d_{D}(c_{t})),
‖x−x′‖=∑t=1,d=1T,D(xt​d−xt​d′)2≤∑t=1,d=1T,Dϵ~2T​D=ϵ\displaystyle\|x-x^{\prime}\|=\sum_{t=1,d=1}^{T,D}(x_{td}-x^{\prime}_{td})^{2}\leq\sum_{t=1,d=1}^{T,D}\frac{\tilde{\epsilon}^{2}}{TD}=\epsilon (10)
fϵ​(𝐱)=(c1,…,cNϵ),ct∈𝒱,f_{\epsilon}(\mathbf{x})=(c_{1},\dots,c_{N_{\epsilon}}),\ c_{t}\in\mathcal{V},
xt​d=(∑n=1Ldn​(ut​d)Mn−12)​K~.\displaystyle x_{td}=\left(\sum_{n=1}^{L}\frac{d_{n}(u_{td})}{M^{n}}-\frac{1}{2}\right)\tilde{K}. (11)
[u:dn(u)=vn,n=1,…,L]=[∑n=1LvnMn,∑n=1LvnMn+1ML)\displaystyle\left[u:\ d_{n}(u)=v_{n},\ n=1,\dots,L\right]=\left[\sum_{n=1}^{L}\frac{v_{n}}{M^{n}},\sum_{n=1}^{L}\frac{v_{n}}{M^{n}}+\frac{1}{M^{L}}\right) (12)
|xt​d−xt​d′|\displaystyle|x_{td}-x_{td}^{\prime}| =|(∑n=1Ldn​(ut​d)Mn−12)​K~−(∑n=1Ldn​(ut​d′)Mn−12)​K~|=|(∑n=1Ldn​(ut​d)Mn−∑n=1Ldn​(ut​d′)Mn)​K~|\displaystyle=\left|\left(\sum_{n=1}^{L}\frac{d_{n}(u_{td})}{M^{n}}-\frac{1}{2}\right)\tilde{K}-\left(\sum_{n=1}^{L}\frac{d_{n}(u_{td}^{\prime})}{M^{n}}-\frac{1}{2}\right)\tilde{K}\right|=\left|\left(\sum_{n=1}^{L}\frac{d_{n}(u_{td})}{M^{n}}-\sum_{n=1}^{L}\frac{d_{n}(u_{td}^{\prime})}{M^{n}}\right)\tilde{K}\right|
≤|M−L​K~|=|M−⌈logM⁡T​D​K~​ϵ~−1⌉​K~|≤|M−logM⁡T​D​K~​ϵ~−1​K~|≤|ϵ~T​D|\displaystyle\leq\left|M^{-L}\tilde{K}\right|=\left|M^{-\lceil\log_{M}\sqrt{TD}\tilde{K}\tilde{\epsilon}^{-1}\rceil}\tilde{K}\right|\leq\left|M^{-\log_{M}\sqrt{TD}\tilde{K}\tilde{\epsilon}^{-1}}\tilde{K}\right|\leq\left|\frac{\tilde{\epsilon}}{\sqrt{TD}}\right|

Appendix B Proof for Proposition 1

Choose any constant latent vector. Map every observation to it and every encoded state–action pair back to it. The latent reconstruction loss is then zero, a global minimum of this nonnegative unregularized objective.

Appendix C Proof for Proposition 2

The proposed proof changes the representation and prediction for two examples so that a fixed decoder reproduces their distinct targets, leaving other outputs unchanged. That construction assumes suitable decoder preimages and independently adjustable inputs.

Those conditions are stronger than merely having two distinct next observations. A constant fixed decoder cannot improve at all, and incompatible targets for the same input cannot both be fitted by the deterministic predictor. Reconstruction can discourage collapse without the unconditional guarantee stated in the proposition.

Equations
(o1,a1,o2),(o3,a3,o4)∈𝒟​ with ​o2≠o4.(o_{1},a_{1},o_{2}),\quad(o_{3},a_{3},o_{4})\in\mathcal{D}\text{ with }o_{2}\neq o_{4}.
ℒgen​(h′,f′,g)=𝔼(o,a,o′)∼𝒟​[‖g⁡(c)−o′‖].\mathcal{L}_{\text{gen}}(h^{\prime},f^{\prime},g)=\mathbb{E}_{(o,a,o^{\prime})\sim\mathcal{D}}\left[\left\lVert g(c)-o^{\prime}\right\rVert\right].
ℒgen​(h~,f~,g)<ℒgen​(h′,f′,g)\mathcal{L}_{\text{gen}}(\tilde{h},\tilde{f},g)<\mathcal{L}_{\text{gen}}(h^{\prime},f^{\prime},g)

Appendix D Proof of Theorem 2

The proof tries to compare Gaussian KL divergences using data processing, then apply round-trip consistency and the triangle inequality. Its key step treats an encoded observation Gaussian as a unit-covariance latent Gaussian. A general encoder does not preserve that covariance or even Gaussianity.

Consequently, the claimed bound and equality under exact inversion do not follow. The scalar scaling counterexample in Section 4.4 already violates both despite a perfect encoder–decoder round trip. Additional metric or distributional assumptions would be needed.

Equations 13–14
Latent Prediction ​ph∘f:\displaystyle\text{Latent Prediction }p_{h\circ f}:\ s^′|o,a∼𝒩⁡(f⁡(h⁡(o),a),I)\displaystyle\hat{s}^{\prime}\mid o,a\sim\mathcal{N}(f(h(o),a),I)
Latent Target ​q^h:\displaystyle\text{Latent Target }\hat{q}_{h}:\ s^′|o′∼𝒩⁡(h⁡(o′),I)\displaystyle\hat{s}^{\prime}\mid o^{\prime}\sim\mathcal{N}(h(o^{\prime}),I)
Observation Prediction ​ph∘f∘g:\displaystyle\text{Observation Prediction }p_{h\circ f\circ g}:\ o^′|o,a∼𝒩⁡(g∘f⁡(h⁡(o),a),I)\displaystyle\hat{o}^{\prime}\mid o,a\sim\mathcal{N}(g\circ f(h(o),a),I)
ℒlatent\displaystyle\mathcal{L}_{\text{latent}} =‖f⁡(h⁡(o),a)−h⁡(o′)‖=2DKL(q^h(s^′∣o′)∥ph∘f(s^′∣o,a)),\displaystyle=\|f(h(o),a)-h(o^{\prime})\|=\sqrt{2D_{\text{KL}}(\hat{q}_{h}(\hat{s}^{\prime}\mid o^{\prime})\,\|\,p_{h\circ f}(\hat{s}^{\prime}\mid o,a))},
ℒgen\displaystyle\mathcal{L}_{\text{gen}} =‖g∘f⁡(h⁡(o),a)−o′‖=2DKL(q^(o′)∥ph∘f∘g(o′∣o,a)).\displaystyle=\|g\circ f(h(o),a)-o^{\prime}\|=\sqrt{2D_{\text{KL}}(\hat{q}(o^{\prime})\,\|\,p_{h\circ f\circ g}(o^{\prime}\mid o,a))}.
Roundtrip Latent Prediction ​ph∘f∘g∘h:\displaystyle\text{Roundtrip Latent Prediction }p_{h\circ f\circ g\circ h}:\ s^′|o,a∼𝒩⁡(h∘g∘f⁡(h⁡(o),a),I).\displaystyle\hat{s}^{\prime}\mid o,a\sim\mathcal{N}(h\circ g\circ f(h(o),a),I).
DKL(q^h(s^′∣o′)∥ph∘f∘g∘h(s^′∣o,a))≤DKL(q^(o′)∥ph∘f∘g(o′∣o,a)),D_{\text{KL}}(\hat{q}_{h}(\hat{s}^{\prime}\mid o^{\prime})\,\|\,p_{h\circ f\circ g\circ h}(\hat{s}^{\prime}\mid o,a))\leq D_{\text{KL}}(\hat{q}(o^{\prime})\,\|\,p_{h\circ f\circ g}(o^{\prime}\mid o,a)),
‖h∘g∘f⁡(h⁡(o),a)−h⁡(o′)‖≤ℒgen.\|h\circ g\circ f(h(o),a)-h(o^{\prime})\|\leq\mathcal{L}_{\text{gen}}. (13)
‖h∘g∘f⁡(h⁡(o),a)−f⁡(h⁡(o),a)‖≤ϵ.\|h\circ g\circ f(h(o),a)-f(h(o),a)\|\leq\epsilon. (14)
ℒlatent\displaystyle\mathcal{L}_{\text{latent}} =‖f⁡(h⁡(o),a)−h⁡(o′)‖\displaystyle=\|f(h(o),a)-h(o^{\prime})\|
≤‖f⁡(h⁡(o),a)−h∘g∘f⁡(h⁡(o),a)‖⏟≤ϵ​ by Inequality 14+‖h∘g∘f⁡(h⁡(o),a)−h⁡(o′)‖⏟≤ℒgen​ by Inequality 13\displaystyle\leq{\underbrace{\textstyle\|f(h(o),a)-h\circ g\circ f(h(o),a)\|}_{\leq\epsilon\text{ by Inequality~\ref{ineq:roundtrip-consistency}}}}+{\underbrace{\textstyle\|h\circ g\circ f(h(o),a)-h(o^{\prime})\|}_{\leq\mathcal{L}_{\text{gen}}\text{ by Inequality~\ref{ineq:data-processing-inequality}}}}
≤ℒgen+ϵ\displaystyle\leq\mathcal{L}_{\text{gen}}+\epsilon
12​ℒgen2\displaystyle\frac{1}{2}\mathcal{L}_{\text{gen}}^{2} =DKL(q^(o′)∥ph∘f∘g(o′∣o,a))\displaystyle=D_{\text{KL}}(\hat{q}(o^{\prime})\,\|\,p_{h\circ f\circ g}(o^{\prime}\mid o,a))
=DKL(q^h(s^′∣o′)∥ph∘f∘g∘h(s^′∣o,a))\displaystyle=D_{\text{KL}}(\hat{q}_{h}(\hat{s}^{\prime}\mid o^{\prime})\,\|\,p_{h\circ f\circ g\circ h}(\hat{s}^{\prime}\mid o,a))
=12​‖h∘g∘f⁡(h⁡(o),a)−h⁡(o′)‖2\displaystyle=\frac{1}{2}\|h\circ g\circ f(h(o),a)-h(o^{\prime})\|^{2}
=12∥f(h(o),a)−h(o′)∥2(h∘g(s^)=s^)\displaystyle=\frac{1}{2}\|f(h(o),a)-h(o^{\prime})\|^{2}\quad(h\circ g(\hat{s})=\hat{s})
=12​ℒlatent2.\displaystyle=\frac{1}{2}\mathcal{L}_{\text{latent}}^{2}.
Concept demonstration

Open image ↗