D-JEPA: A Decision-Aligned Latent World Model
Shuaijun Liu, Chengyu Wu, Qifu Wen, Feiyang You, Chenglong Zhang, Shuyang Hao, Xi Lin, Ningxin Su
The Hong Kong University of Science and Technology (Guangzhou) · Boston University · Shanghai Jiao Tong University
Published September 21, 2026 · on the feed September 28, 2026 · ▲ 17 · arXiv 2609.24749
The abstract
Latent world models predict the consequences of actions, but accurate prediction does not guarantee that latent distance reflects which candidate will execute successfully. We identify a decision-local prediction gap: among the few futures competing for execution, a candidate predicted closer to the goal can produce a worse realized outcome than an available alternative. We introduce D-JEPA, a decision-aligned latent world model that learns decision-relevant relations among candidate futures from executed outcomes. A bounded, permutation-equivariant operator jointly reasons over goal-relative predictive features and ordinal evidence, refining pretrained predictive geometry where action choices are most consequential. Restricted predictor adaptation and a shared ordinal interface extend this alignment across complementary predictive geometries. D-JEPA further realizes the learned decision structure in JEPA-compatible future representations, enabling deployment through native latent-distance planning. Evaluations across latent control, manipulation, pretrained action-producing models, physical robots and autonomous driving demonstrate improved action selection, including 87.89% success on PushT, a 15.04-point average gain on RoboTwin, and a 17-point gain on physical robot tasks. These results establish decision-relevant relational structure as a direct bridge between predictive world modeling and effective control.
The Brief
2:22A short podcast summary of the paper. Written and voiced by AI.
Transcript
D-JEPA. A Decision-Aligned Latent World Model. By Shuaijun Liu and seven others, from the Hong Kong University of Science and Technology, Guangzhou, Boston University, and one other institution. Published on the twenty-first of September. Its code repository is worked out: about one hundred and sixty source files, with a readme and a licence.
A latent world model predicts what an action will lead to, and a planner can then pick the candidate whose predicted future sits closest to the goal. The trouble is that predicting accurately does not guarantee that distance in the latent space reflects which candidate will actually execute successfully. The authors identify a specific form of this, which they call a decision-local prediction gap. Among the few futures competing for execution, a candidate predicted to land closer to the goal can produce a worse realized outcome than an alternative that was available. Near the point of decision, closer in prediction can mean worse in practice.
D-JEPA is a decision-aligned latent world model. Rather than relying on prediction alone, it learns the decision-relevant relations among candidate futures from outcomes that were actually executed. The aim is to correct the geometry exactly where choosing between actions matters most.
At its core is an operator that is bounded and permutation-equivariant. It reasons jointly over two things: predictive features measured relative to the goal, and ordinal evidence. With these it refines the pretrained predictive geometry in the places where action choices are most consequential. Restricted adaptation of the predictor, together with a shared ordinal interface, extends this alignment across complementary predictive geometries. Finally, the learned decision structure is realized in future representations that remain compatible with JEPA, which means the model can be deployed through the same native latent-distance planning.
Evaluations cover latent control, manipulation, pretrained action-producing models, physical robots and autonomous driving, and show improved action selection throughout. D-JEPA reaches eighty-seven point eight nine percent success on PushT. It gains fifteen point zero four points on average on RoboTwin. And on physical robot tasks it gains seventeen points.
The authors read these results as establishing relational structure among decisions as a direct bridge from predictive world modeling to effective control. Put simply, it is not enough for a world model to foresee the future well. The ordering it puts on the handful of futures a robot is choosing between has to be right too.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Your planner predicts well, and still picks the wrong action. Start there.
You plan with a world model that works in a learned feature space. For each candidate action it predicts the future, and you execute the one whose predicted future sits closest to the goal.
The road ahead
- 1 · The problem ~7 min A world model can predict well overall yet still rank the last few competing actions wrongly, so the closest-looking choice can fail.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min D-JEPA learns, from really executing the candidates, small bounded corrections to their ranking, then writes that order back into the predicted futures.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Refine the pretrained ranking with executed outcomes, write that order into the futures, then plan by plain nearest-to-goal distance.
- 6 · The experiment ~6 min You’ll guess: How many points did D-JEPA, a decision-aligned latent world model, gain on physical robot tasks?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.