At a glance
Intuition. Two locations can look close in a picture while a wall makes the route between them long. A useful planning representation should reflect the effort of reaching the goal, rather than visual similarity alone. This paper trains a JEPA encoder so that latent distance approximates the negative of a goal-conditioned value function. An asymmetric quasi-distance can represent situations where going from A to B differs from returning from B to A. The learned dynamics model then supports model-predictive control in this reshaped space. Experiments on wall and maze tasks favor the value-guided objectives, especially the quasi-distance variant, over prediction-only and simple geometric objectives. The evidence is from small simulated settings. A good value fit, global planning reliability, and robustness to stochastic transitions remain separate questions.
1 Introduction
The proposed intervention changes the geometry used to score imagined plans. A prediction loss can learn a useful state representation without making distance-to-goal an easy optimization landscape. Training latent distance to approximate a reaching value function aims to make that distance decrease along useful routes. Planning still requires an action-conditioned dynamics predictor and an optimizer; the encoder does not itself become a policy.
2 Related work
2.1 JEPA world models
A JEPA predicts future encoded states from the current state and actions. Variance/covariance regularization or a slowly updated target encoder can discourage trivial constant representations. At execution time, model-predictive control repeatedly searches for an action sequence whose predicted endpoint resembles the encoded goal, executes an initial portion, and replans. The paper targets the quality of that resemblance measure.
2.2 Learning a value function
A goal-conditioned value estimates the discounted cost of reaching a particular goal. Implicit Q-learning learns this information from offline trajectories using expectile regression. Euclidean distance imposes symmetry, whereas reaching costs can be directional. A quasi-distance relaxes symmetry while retaining useful distance-like structure, motivating the two representation families compared here.
3 Value-guided JEPA for action planning
The encoder is trained to make negative latent distance behave like a reaching value. This supplies the planning optimizer with information about routes and obstacles that plain appearance distance can miss. The mechanism is a learned objective geometry, not an extra terminal-value network attached only at deployment.
3.1 Baseline loss functions
Two simpler alternatives test whether local geometric constraints suffice: contrastive learning pulls consecutive states together relative to unrelated pairs, and regression encourages successive-state distance to equal one. These objectives describe local transitions but do not explicitly solve the discounted goal-reaching value problem.
3.2 IQL for JEPA models
The value is parameterized as negative encoded-state distance. A discounted Bellman-style target and asymmetric squared expectile loss favor favorable transitions supported by the dataset. Goals include trajectory endpoints and randomly sampled batch states. The authors compare Euclidean and learned quasi-metric distances, and either pretrain the encoder before fitting dynamics or optimize representation and prediction together. The discount and expectile coefficients control both long-range information and training stability.
Equation 1
| (1) |
4 Experiments
4.1 Experiment settings
Offline random trajectories supply all training data. In Wall, an agent must reach a point across a wall through its door; small- and large-step datasets differ in exploration. In Maze, velocity commands and inertia make a single image insufficient, so velocity is supplied to the encoder. Evaluation uses maze layouts distinct from the training layouts.
4.2 Planning with the representations
The retained tables compare prediction, geometric, and value-based representation objectives under the same planning procedure. Value-guided approaches generally improve success, and quasi-distance is strongest in the reported settings. Jointly adding prediction or variance/covariance regularization is not automatically beneficial. Larger-step wall data also helps, illustrating that the objective and dataset jointly determine the learned planning geometry.
| Name | State encoder loss | Sep |
|---|---|---|
| VF_pred | ||
| VF_quasi | & quasi-distance | ✓ |
| VF_quasi_pred | & quasi-distance | |
| VF_VCReg | & | ✓ |
| VF_VCReg_pred | & |
5 Discussion
Distant state–goal relationships are sparsely sampled, and discounted values provide weak gradients far from the goal. Thus local accuracy need not produce a reliable global route map. Hierarchical representations are suggested as one remedy. Trajectory quality also matters: highly suboptimal paths can distort apparent reaching costs, while narrowly expert data can lose coverage. Neither more data nor expert demonstrations alone resolves every tradeoff.
6 Conclusion
Value-guided representation learning is a promising complement to JEPA prediction for these deterministic control tasks. The main result is improved planning success, not a general replacement for predictive learning. Stochastic environments are an important limitation because the implicit-Q-learning objective can be biased there; broader evaluations are left open.
7 Appendix
7.1 Datasets
7.1.1 Wall
Wall observations separate agent and obstacle information into two image channels. The datasets contain 1,000 trajectories of length 64, with half crossing a doorway. Persistent random directions encourage broader movement than independent Gaussian actions. The larger-step dataset explores more space per trajectory, so its better performance should be interpreted together with this coverage difference.
7.1.2 Maze
Maze uses randomized connected layouts and velocity-controlled motion. Each training set has 1,000 trajectories of length 101 across five layouts, while evaluation uses different layouts and well-separated start–goal pairs. Supplying velocity addresses partial observability from inertia; the evaluation therefore is not vision-only state inference.
7.2 Experiment settings
The implementation uses 512-dimensional state representations, a convolutional residual encoder, and a multilayer-perceptron predictor. The appendix preserves optimization settings and the distinct model-predictive path-integral planning budgets for Wall and Maze. Euclidean and quasi-distance variants use separately selected discount and expectile settings, which matters when attributing differences solely to representation geometry.
7.3 Additional experiments
Increasing the discount or expectile can strengthen distant-goal information until instability reverses the gain. Parameters are chosen on a separate wall dataset. The additional study considers separating predictive and planning representations rather than assuming one space must optimize both objectives equally well. These ablations qualify the main claim: useful planning geometry requires tuning, and remains an empirical approximation.