Literature
Display
← All papers

Causal-JEPA: Learning World Models through Object-Level Latent Masking

Heejeong Nam · Quentin Le Lidec · Lucas Maes · Yann LeCun · Randall Balestriero

Preprint posted · ICML 2026 (accepted) ↗

World models12 min text · CondensedOriginal paper ↗
Contents

Overview

The pipeline shows frozen object perception, history masking and interaction-dependent prediction. The reading notes below explain the mechanism before separating reasoning and control evidence.

Refer to caption
Figure 1: C-JEPA training pipeline. A frozen encoder extracts object-centric representations, followed by selective masking across history. The predictor recovers masked history slots and predicts future latent states, conditioned on optional auxiliary variables, via a joint masked-history and forward-prediction objective.

At a glance

Intuition. If a model can see a ball’s entire recent trajectory, predicting that ball again may never require learning which other ball hit it. Causal-JEPA hides selected objects across the history as well as the future, making the predictor use other objects and observed interactions. Frozen object-centric encoders supply slots; training reconstructs masked object latents, without a pixel-reconstruction decoder. The paper compares this masking objective with otherwise similar unmasked models on synthetic visual reasoning and Push-T control. Improvements are strongest at suitable masking budgets, while excessive masking can remove the evidence needed for prediction. The causal interpretation is an inductive bias under explicit representation and dynamics assumptions, not discovery of arbitrary real-world causal graphs.

1 Introduction

Object slots alone do not force interaction reasoning: a predictor can extrapolate each visible object independently. The proposed history masking suppresses this shortcut and asks other observed entities to explain the hidden trajectory. The work studies both reasoning quality and efficient control, while holding perception and much of the predictor architecture fixed for key comparisons.

2 Related Works

2.1 Structured World Models for Dynamics Learning

Structured world models divide scenes into objects, relations, or other entities. This can simplify dynamics but still leaves shortcut solutions that ignore interactions. The paper positions masking as an objective-level complement to object-centric architectures, rather than an alternative perception mechanism.

2.2 Masking-Based Representation Learning

Patch, tube, and latent masking have made self-supervised prediction useful across images and video. Here the hidden unit is an object slot across time. That distinction is intended to remove object-specific self-prediction while leaving interaction evidence in the visible context.

3 Preliminaries

Joint Embedding Predictive Architecture

A JEPA predicts encoded future observations from past observations and actions. Encoders and latent predictors separate representation from prediction, avoiding pixel reconstruction. This work keeps its perception model frozen so that the main experiment isolates what the predictor’s masking objective contributes.

Object-Centric Representation via Slot Attention

Slot attention summarizes a scene with a fixed number of object-like vectors. Slots are useful only insofar as they track coherent entities across the relevant history. Their quality is an assumption inherited by the masking method, not something the predictor automatically repairs.

4 Method

The method first specifies visible object histories and optional actions or proprioception, then predicts masked history and future slots. The pipeline and masking diagram are preserved with the original captions.

4.1 Problem Setting

An encoded history is a sequence of object slots, with a fixed prediction horizon and optional observed variables. The task is to infer missing object states and their continuation. Object identity and time must remain distinguishable even when the actual state is hidden.

Equation 1
St=g⁡(Xt)={st1,…,stN},sti∈ℝd,S_{t}=g(X_{t})=\{s_{t}^{1},\dots,s_{t}^{N}\},\qquad s_{t}^{i}\in\mathbb{R}^{d}, (1)

Masked and Visible Object Sets.

Selected objects are hidden across the observation history while a minimal identity anchor remains. Visible objects supply context. Hiding trajectories rather than isolated tokens makes it harder to solve the task by copying a nearby state of the same object.

Equation 2
Sτm={sτi∣i∈ℳτ},Sτc={sτj∣j∉ℳτ}.S_{\tau}^{\mathrm{m}}=\{s_{\tau}^{i}\mid i\in\mathcal{M}_{\tau}\},\qquad S_{\tau}^{\mathrm{c}}=\{s_{\tau}^{j}\mid j\notin\mathcal{M}_{\tau}\}. (2)

Auxiliary Observable Variables.

Actions and proprioceptive signals can be appended as auxiliary observed entities. They provide control information without forcing every object slot to carry it. The integration choices matter in action-conditioned planning and are compared separately.

4.2 Causal-JEPA

A causal transformer receives the masked entity sequence, with learned mask and temporal information. It learns to recover hidden slots and predict later ones using the observable context.

Refer to caption
Figure 2: Object-level latent masking in C-JEPA. Selected object slots are masked across time, except for a minimal identity anchor, forcing the predictor to infer object dynamics from interactions with other objects and auxiliary variables.

Object-Level Masking for Learning Interaction.

The key intervention is on predictor access: remove the target object’s directly observable history and force dependence on informative interactions. This is a latent masking intervention on what the learner sees, not a physical intervention that changes the data-generating world.

Equation 3
z~τi=ϕ⁡(zt0i)+eτ,\tilde{z}_{\tau}^{i}=\phi(z_{t_{0}}^{i})+e_{\tau}, (3)

Learning Objective.

The objective combines reconstruction of masked history slots and prediction of future slots, using frozen encoder outputs as targets. It excludes a pixel decoder so that the representation-learning comparison remains reconstruction-free. Loss weights and masking budgets determine how much inference the predictor must perform.

Equations 4–6
Z^𝒯=f⁡(Z¯𝒯).\hat{Z}_{\mathcal{T}}=f\big(\bar{Z}_{\mathcal{T}}\big). (4)
ℒmask=𝔼[∑τ∈𝒯∑i=1N[z¯τi≠zτi]‖z^τi−zτi‖22],\mathcal{L}_{\mathrm{mask}}=\mathbb{E}\!\left[\sum_{\tau\in\mathcal{T}}\sum_{i=1}^{N}\mathbf{1}\!\left[\bar{z}_{\tau}^{i}\neq z_{\tau}^{i}\right]\,\left\|\hat{z}_{\tau}^{i}-z_{\tau}^{i}\right\|_{2}^{2}\right], (5)
ℒmask=\displaystyle\mathcal{L}_{\mathrm{mask}}\;= 𝔼[‖z^τi−zτi‖22|i∈ℳ,τ≤t]⏟ℒhistory\displaystyle\underbrace{\mathbb{E}\!\left[\left\|\hat{z}_{\tau}^{i}-z_{\tau}^{i}\right\|_{2}^{2}\;\middle|\;i\in\mathcal{M},\ \tau\leq t\right]}_{\mathcal{L}_{\mathrm{history}}} (6)
+𝔼⁡[‖Z^τ−Zτ‖22|τ>t]⏟ℒfuture.\displaystyle+\underbrace{\mathbb{E}\!\left[\left\|\hat{Z}_{\tau}-Z_{\tau}\right\|_{2}^{2}\;\middle|\;\tau>t\right]}_{\mathcal{L}_{\mathrm{future}}}.

Inference.

At deployment, the history is fully visible and only future tokens remain unavailable. The model therefore uses its learned interaction structure on a conventional prediction task; training-time masking is not a requirement to discard observations during control.

5 Experiments

Two evaluations test different claims: visual question answering probes predicted interactions, while Push-T probes action-conditioned planning efficiency. Their scores should not be treated as interchangeable evidence of a general causal world model.

5.1 Causal-JEPA for Visual Reasoning

Task and Evaluation.

CLEVRER contains synthetic collision videos and descriptive, predictive, explanatory, and counterfactual questions. ALOE answers questions using predicted object trajectories. Consequently, the evaluation tests a predictor-plus-reasoner pipeline rather than direct language generation by Causal-JEPA.

Baseline Models.

SlotFormer, OCVP-Seq, and the unmasked OC-JEPA comparison use the same object-centric perception where applicable. Removing reconstruction from older models can itself hurt them, so both their original and reconstruction-free results are important controls.

Results.

Masking improves the controlled unmasked comparison, but the best budget differs by encoder. With SAVi, two masked slots improve average question accuracy from 77.28 to 83.88 and counterfactual per-question accuracy from 41.10 to 60.19. Larger budgets can reverse the gain. The complete tables are restored from the exact-version PDF because the source HTML represents them as nested spans.

Table 1. VQA accuracy under varying numbers of masked objects. Overall accuracy is reported per question, with counterfactual ac- curacy additionally reported per option and per question. (V) uses VideoSAUR and (S) uses SAVi encoders. Performance changes relative to the unmasked baseline are indicated by arrows.
Table 1. VQA accuracy under varying numbers of masked objects. Overall accuracy is reported per question, with counterfactual ac- curacy additionally reported per option and per question. (V) uses VideoSAUR and (S) uses SAVi encoders. Performance changes relative to the unmasked baseline are indicated by arrows. Restored as vector artwork from page 5 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.
Table 2. Baseline comparison using a SAVi encoder. For models with and without reconstruction, arrows indicate differences rela- tive to the reconstruction-based variant.
Table 2. Baseline comparison using a SAVi encoder. For models with and without reconstruction, arrows indicate differences rela- tive to the reconstruction-based variant. Restored as vector artwork from page 6 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.

5.2 Causal-JEPA for Efficient World Modeling

Task and Evaluation.

Push-T planning selects actions whose predicted final object state matches a goal image. Success tests contact-rich manipulation in simulation. This setting evaluates both compactness and usefulness of the learned dynamics, not just predictive image similarity.

Equation 7
at:t+H−1∗≜arg​minat:t+H−1‖S^t+H−Sg‖22,a_{t:t+H-1}^{*}\triangleq\argmin_{a_{t:t+H-1}}\left\lVert\hat{S}_{t+H}-S_{g}\right\rVert_{2}^{2}, (7)

Baseline Models.

Patch-based DINO-WM, a register variant, object-centric DINO-WM, and unmasked OC-JEPA separate perception, token budget, predictive objective, and masking. These distinctions prevent attributing every difference to object-centric representation alone.

Results.

Causal-JEPA achieves 88.67% success using six 128-dimensional slots in the reported comparison, versus 91.33% for the much larger patch-token DINO-WM representation. The masked objective substantially improves the compact unmasked baseline. This is an efficiency/accuracy tradeoff, not a strict win over the strongest patch model.

Table 3. Push-T planning success rates across world models and token budgets. Changes are reported relative to OC-DINO-WM, and |M| = 1 is used for C-JEPA.
Table 3. Push-T planning success rates across world models and token budgets. Changes are reported relative to OC-DINO-WM, and |M| = 1 is used for C-JEPA. Restored as vector artwork from page 6 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.

Action and Proprioception as Auxiliary Nodes.

Explicit action and proprioception nodes improve access to relevant control information. Their placement changes what the transformer can condition on; the architecture diagrams distinguish these auxiliary channels from the masked object states.

Refer to caption
Figure 3: Comparison of auxiliary variable integration methods.

5.3 Further Analysis of the Masking Objective

Qualitative analysis on PHYRE.

Selected PHYRE rollouts suggest more plausible interaction predictions, and cross-slot attention is inspected as a dependency proxy. Attention visualizations alone do not establish causal direction or identify an intervention effect.

Ablation on Masking Strategy.

Object, token, and temporal-tube masking remove different evidence. Their budgets must be interpreted in their own units; hiding one object is not identical to hiding an arbitrary equal percentage of tokens.

6 Theoretical Perspective: Causal Inductive Bias of Latent Masking

The formal section explains why hiding a self-history can force useful cross-object dependencies under stated assumptions. It does not claim unrestricted causal identification from passive video.

6.1 Assumption

The analysis assumes a finite object system, shared transition mechanism, object-aligned slots, and a sufficient finite observation history. These conditions make a masked object’s future inferable from remaining relevant context. If perception mixes identities or dynamics depend on hidden unobserved causes, the interpretation can fail.

Note.

The assumptions describe a structured family of environments and representations, not properties guaranteed by using slot attention. They should be checked when transferring the method beyond the synthetic and manipulation domains studied.

6.2 Interaction-Inducing Inductive Bias

Under the model assumptions, a predictor that must recover a masked trajectory cannot rely solely on that object’s observed past. The argument favors interaction-relevant conditional information and suppresses an unmasked shortcut. It establishes an inductive-bias result under observability and representation assumptions, not recovery of a unique physical causal graph.

Equations 8–10
p⁡(zti∣ZT(−i))=p⁡(zti∣𝒩t​(i)).p\!\left(z_{t}^{i}\mid Z_{T}^{(-i)}\right)=p\!\left(z_{t}^{i}\mid\mathcal{N}_{t}(i)\right). (8)
ℒhistoryi:=𝔼⁡[‖z^ti−zti‖22],\mathcal{L}_{\mathrm{history}}^{i}:=\mathbb{E}\!\left[\left\|\hat{z}_{t}^{i}-z_{t}^{i}\right\|_{2}^{2}\right], (9)
z^ti∗=𝔼[zti|ZT(−i)]=𝔼[zti|𝒩t(i)].\hat{z}_{t}^{i\,*}=\mathbb{E}\!\left[z_{t}^{i}\;\middle|\;Z_{T}^{(-i)}\right]=\mathbb{E}\!\left[z_{t}^{i}\;\middle|\;\mathcal{N}_{t}(i)\right]. (10)

7 Conclusion

Object-level history masking is a lightweight objective change that improves several controlled reasoning and control comparisons. Encoder quality and the amount of information removed remain important limitations. The contribution is to interaction-aware prediction with compact object latents, with broader causal claims bounded by the formal assumptions.

Impact Statement

The impact statement discusses possible applications and limitations of learning structured physical dynamics. The experiments do not themselves validate safety in open-world deployment.

Appendix A Notations

The notation table defines object slots, masking sets, history and prediction windows, and auxiliary observed variables. Its complete vector rendering is restored from the PDF so the symbols in the theoretical and implementation sections remain available.

Table A1. Notations used throughout the paper.
Table A1. Notations used throughout the paper. Restored as vector artwork from page 13 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.

Appendix B Discussion

B.1 Interpretation of Object-Level Masking as a Latent Intervention on Predictor Observability

Masking replaces an object’s directly accessible latent state with an uninformative token while keeping selected context. This intervenes on predictor observability. It need not correspond to a do-intervention on the physical object or modify the true transition mechanism.

Equation
Z¯t=(Zt∖{zti})∪{z~ti},\bar{Z}_{t}\;=\;\big(Z_{t}\setminus\{z_{t}^{i}\}\big)\cup\{\tilde{z}_{t}^{i}\},

Discussion.

The interpretation is useful because it specifies which shortcut is removed: direct access to a target’s own state. Any resulting dependence on other slots remains conditional on what the encoder and visible history reveal.

B.2 Relation to Classical Causal Concepts

The appendix relates this training bias to familiar causal concepts while separating their stronger identification requirements from the actual masking objective.

Relation to Markov blankets and causal parents.

Predictively informative neighboring slots can resemble a Markov-blanket context, but the model is not given a known causal graph or proved to recover all parents. A learned conditional dependency and a causally sufficient parent set are distinct objects.

Relation to invariant causal prediction and invariant risk minimization.

Changing which information is visible can discourage reliance on unstable shortcuts, echoing invariant-prediction motivations. The masking configurations are not automatically distinct physical environments satisfying invariant-causal-prediction assumptions.

Summary.

The justified conclusion is a bias toward interaction-relevant predictive structure. Stronger causal identification requires assumptions or interventions beyond the masking recipe itself.

Appendix C Dataset

C.1 CLEVRER ( Yi et al., 2020 )

CLEVRER supplies synthetic collision videos with multiple question types and held-out examples. Frozen object encoders and the downstream question-answering system introduce additional components whose settings are retained for reproducing the comparison.

C.2 Push-T ( Chi et al., 2025 )

Push-T supplies action-conditioned manipulation trajectories and goal configurations. Its small set of coherent objects makes compact slots attractive, while contact events require information about interactions rather than independent object extrapolation.

Appendix D Encoders

Refer to caption
Figure A1: Slot visualization from object-centric encoders.

D.1 VideoSAUR ( Zadaianchuk et al., 2023 )

VideoSAUR supplies frozen object-centric video representations. Its training and slot structure determine what the JEPA predictor receives; improvements attributed to masking are conditional on this perception stage.

Common Setup.

The common encoder setup uses pretrained visual features and learned object grouping before predictor training. Keeping it fixed helps isolate objective changes but does not establish that the predictor can recover missing or incorrectly grouped objects.

Training Details on CLEVRER.

The appendix gives the CLEVRER-specific encoder training schedule and configuration. These are perception-stage settings, separate from masked latent prediction and question-answering training.

Training Details on Push-T.

Push-T uses its own encoder configuration and training data. Results therefore should be compared within each task’s controlled setup rather than assuming one unchanged perception model spans every experiment.

D.2 SAVi ( Kipf et al., 2022 )

SAVi is the alternative frozen object encoder for testing whether the masking benefit depends on a single representation family.

Training Details on CLEVRER.

The SAVi training recipe and slot choices are retained. Differences from VideoSAUR affect both the attainable accuracy and the best masking budget, as the complete result tables show.

D.3 Auxiliary Variable Encoders.

Small encoders project auxiliary actions and proprioception into tokens compatible with the predictor. These channels supplement, rather than replace, object-slot observations.

Appendix E Predictors

E.1 Design Choice

A transformer can attend across the visible entities and times, unlike separate independent per-object predictors. The chosen architecture makes cross-object inference possible; the masking objective creates pressure to use it.

E.2 Implementation Details

Temporal embeddings, mask embeddings, token projection and causal attention specify how unavailable history and future states are represented. These details are necessary to reproduce the intended information boundary.

E.3 Training Details

The appendix retains prediction horizon, masking and optimization choices for each domain. The encoder remains frozen, so these settings train latent dynamics rather than a new end-to-end visual representation.

Appendix F Visual Question Answering

F.1 ALOE ( Ding et al., 2021 )

ALOE converts object trajectories and question tokens into visual-question-answering predictions. Its scores assess how useful those trajectories are to the reasoner, not only whether the dynamics objective was minimized.

Training Details on CLEVRER.

The CLEVRER reasoner setup fixes training and evaluation protocols across the compared predictors. This control is essential because changing the reasoning model could otherwise confound the masking comparison.

Appendix G Model Predictive Control Planning Details

Goal-conditioned model-predictive control encodes the current and target scene, imagines candidate action sequences, and repeatedly executes only an initial portion of the selected plan.

Planning Configuration.

Planning horizons, candidate counts and replanning choices determine test-time compute. They are held to the stated settings when comparing success rates across representation sizes.

Cost Function.

The cost compares predicted goal slots with target slots. It measures task progress only insofar as the frozen encoder and slot matching preserve the relevant object configuration.

Action Optimization via CEM.

Cross-entropy optimization iteratively samples candidate actions, keeps elite sequences and refits the sampling distribution. It does not use a separately trained task policy.

Hungarian Matching for Object-Centric Models.

Hungarian assignment aligns object slots before computing a comparison cost. This handles arbitrary slot ordering but cannot correct an encoder that merges or loses physical objects.

Appendix H Baselines

Refer to caption
Figure A2: Diagram of DINO-WM (Zhou et al., 2025), OC-DINO-WM, and OC-JEPA

SlotFormer ( Wu et al., 2023 )

SlotFormer rolls object states forward autoregressively. Its reconstruction-based and reconstruction-free variants distinguish benefits from a pixel decoder from those of latent prediction alone.

OCVP-Seq ( Villar-Corrales et al., 2023 )

OCVP-Seq explicitly separates self-dynamics from interactions. The controlled variants test whether that architectural factorization suffices when reconstruction is removed.

DINO-WM ( Zhou et al., 2025 ) and DINO-WM with Register ( Darcet et al., 2024 )

DINO-WM uses many frozen visual patch tokens and an action-conditioned autoregressive predictor. The register variant changes perception. Its larger token budget is important when interpreting its high Push-T success.

OC-DINO-WM

Object-centric DINO-WM replaces patch features with object slots while retaining the predictive framework, isolating the representation change.

OC-JEPA

OC-JEPA uses the same general architecture with no history-object masking. It is the most direct control for the proposed objective, rather than a fully unrelated baseline.

Appendix I Full Results

The complete question-type and counterfactual tables qualify the aggregate gains. They show different optimal masking budgets for SAVi and VideoSAUR, and that older reconstruction-based baselines may degrade sharply when reconstruction is removed. Both tables are restored from the matching PDF into readable vector artwork.

Table A2. VQA accuracy under varying numbers of masked objects. (V) uses VideoSAUR and (S) uses SAVi encoders. Performance changes relative to the unmasked baseline are indicated by arrows.
Table A2. VQA accuracy under varying numbers of masked objects. (V) uses VideoSAUR and (S) uses SAVi encoders. Performance changes relative to the unmasked baseline are indicated by arrows. Restored as vector artwork from page 19 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.
Table A3. Baseline comparison using a SAVi encoder. For models with and without reconstruction, arrows indicate differences relative to the reconstruction-based variant.
Table A3. Baseline comparison using a SAVi encoder. For models with and without reconstruction, arrows indicate differences relative to the reconstruction-based variant. Restored as vector artwork from page 20 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.

Appendix J Qualitative Analysis on PHYRE

PHYRE supplies out-of-training-domain multi-object interactions for qualitative inspection. Selected rollouts and attention maps help reveal behavior but do not constitute a comprehensive quantitative generalization benchmark.

Refer to caption
Figure A3: Qualitative rollout analysis on PHYRE. We compare imagined future trajectories produced by OC-JEPA and C-JEPA in representative multi-body interaction cases. Please refer to https://github.com/galilai-group/cjepa/tree/main/dataset/phyre for better visualization.
Refer to caption
Figure A4: Attention-based dependency analysis on PHYRE. We visualize cross-slot attention patterns in the predictor as a qualitative proxy for local temporal interaction structure. Cells marked in red indicate the query token corresponding to the target slot.

Rollout plausibility.

Rollout examples compare imagined collision outcomes under masked and unmasked training. Their value is mechanistic diagnosis; selection and perception quality limit broad conclusions.

Attention-based dependency proxy.

Cross-slot attention is inspected as a proxy for information dependence. Attention weights can highlight which entities the predictor uses, but are not an identified causal graph.

Discussion.

The qualitative cases are consistent with useful interaction reasoning beyond the training domain. They supplement the controlled benchmark comparisons rather than proving unrestricted transfer.

Appendix K Exploring Different Masking Strategies

Token, tube and object masking are compared over several budgets. The full tables show why masking structure and amount must be tuned together; masking more is not monotonically better.

Table A4. Effect of masking strategies and masking budgets on CLEVRER performance. For token- and tube-level masking, the budget is reported as a percentage of masked tokens. For object-level masking (C-JEPA), the budget is reported as the number of masked objects out of a total of seven.
Table A4. Effect of masking strategies and masking budgets on CLEVRER performance. For token- and tube-level masking, the budget is reported as a percentage of masked tokens. For object-level masking (C-JEPA), the budget is reported as the number of masked objects out of a total of seven. Restored as vector artwork from page 23 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.
Table A5. Effect of masking strategies and masking budgets on CLEVRER performance. For token- and tube-level masking, the budget is reported as a percentage of masked tokens. For object-level masking (C-JEPA), the budget is reported as the number of masked objects out of a total of four.
Table A5. Effect of masking strategies and masking budgets on CLEVRER performance. For token- and tube-level masking, the budget is reported as a percentage of masked tokens. For object-level masking (C-JEPA), the budget is reported as the number of masked objects out of a total of four. Restored as vector artwork from page 23 of the matching PDF because the source HTML encodes the table as nested spans rather than a readable table.

Results.

The preferred masking unit depends on the question type and budget, and aggressive masking can destroy necessary context. The retained Table A5 caption names CLEVRER although its four-object percentages align with the control masking comparison; the reading preserves the printed source rather than silently relabeling its values.

Appendix L Assumptions

The formal assumptions constrain object alignment, transition stationarity, available history and how observed entities influence one another. They provide a setting in which deleting self-history forces alternative predictive information. They are not guaranteed by passive video collection, and hidden variables or inconsistent slots can violate them.

Appendix M Theoretical Analysis

M.1 Proof of Theorem 1

The proof conditions on visible context after removing the target object history. Under the stated factorization and sufficiency assumptions, useful recovery must rely on interaction-relevant information rather than direct self-copying. The proof concerns the induced learning problem, not arbitrary intervention identification.

Equations
ℛ⁡(h):=𝔼⁡[‖Y−h⁡(X)‖22].\mathcal{R}(h):=\mathbb{E}\!\left[\|Y-h(X)\|_{2}^{2}\right].
𝔼⁡[‖Y−h⁡(X)‖22∣X]\displaystyle\mathbb{E}\!\left[\|Y-h(X)\|_{2}^{2}\mid X\right] =𝔼⁡[‖Y−m⁡(X)‖22∣X]+‖m⁡(X)−h⁡(X)‖22.\displaystyle=\mathbb{E}\!\left[\|Y-m(X)\|_{2}^{2}\mid X\right]+\|m(X)-h(X)\|_{2}^{2}.
h∗(X)=𝔼[Y∣X]and hencez^ti∗=𝔼[zti∣ZT(−i)],h^{*}(X)=\mathbb{E}[Y\mid X]\quad\text{and hence}\quad\hat{z}_{t}^{i\,*}=\mathbb{E}\!\left[z_{t}^{i}\mid Z_{T}^{(-i)}\right],
p⁡(zti∣ZT(−i))=p⁡(zti∣𝒩t​(i)).p\!\left(z_{t}^{i}\mid Z_{T}^{(-i)}\right)=p\!\left(z_{t}^{i}\mid\mathcal{N}_{t}(i)\right).
𝔼⁡[φ⁡(zti)∣ZT(−i)]=𝔼⁡[φ⁡(zti)∣𝒩t​(i)]a.s.\mathbb{E}\!\left[\varphi(z_{t}^{i})\mid Z_{T}^{(-i)}\right]=\mathbb{E}\!\left[\varphi(z_{t}^{i})\mid\mathcal{N}_{t}(i)\right]\quad\text{a.s.}
𝔼⁡[zti∣ZT(−i)]=𝔼⁡[zti∣𝒩t​(i)],\mathbb{E}\!\left[z_{t}^{i}\mid Z_{T}^{(-i)}\right]=\mathbb{E}\!\left[z_{t}^{i}\mid\mathcal{N}_{t}(i)\right],

M.2 Interpretation of Theorem 1 .

The theorem supports the phrase causal inductive bias in a restricted predictive sense. Recovering masked states is still evaluated through a learned representation and an observational distribution.

M.3 Why object-level trajectory masking suppresses shortcuts.

Masking a whole object trajectory suppresses temporal interpolation of that object from adjacent visible states. A minimal anchor preserves identity while limiting the direct state information available to the predictor.

M.4 Justification of Corollary 1

The corollary follows by applying the theorem to the specified masking configuration and context variables. Its force depends on the same object-alignment and observability conditions; it does not remove those assumptions.

Concept demonstration

Open image ↗