Proving and Improving the Platonic Representation Hypothesis with JEPAs

Proving and Improving the Platonic Representation Hypothesis with JEPAs. By Randall Balestriero and one other, from Brown University, New York University, and one other institution. Published on the twenty-ninth of October.

A central question in AI is whether models trained on different modalities end up with the same representation of the world. The Platonic Representation Hypothesis, or PRH, conjectures that they do, as they scale. But a conjecture without a proof offers neither a guarantee nor a recipe. This work sets out to turn the hypothesis into a theorem, and the setting it chooses is Joint-Embedding Predictive Architectures, known as JEPAs.

The first result is what the authors call the Platonic Representation Theorem. JEPAs trained separately, on any number of modalities, learn the same representation up to a single rotation. That holds whenever their positive pairs are two observations of one scene, which the paper calls good views. When the views are not good in that sense, their misalignment is bounded.

The second step reframes something familiar. Next-token prediction, the paper shows, is itself a JEPA, but one with bad views: its positive pairs are parts of a single document, not two observations of one scene. That reading explains why language models align with vision only through scale. The theorem's condition is not met, so alignment has to be bought some other way.

The third step is empirical validation, running from one hundred forty-five thousand captions up to the texts of nine hundred forty million web images. Along the way the authors uncover a scaling law of alignment that predicts unseen runs to within seed noise. The size of the gap between good and bad views is large. On the same corpus, good views need forty-two times fewer training samples than next-token prediction to reach the same alignment.

A text JEPA with one hundred nine million parameters aligns with the image encoders of DINOv2 and CLIP better than every language model evaluated, up to seventy-two billion parameters, and better than CLIP's own text encoder. The authors conclude that next-token prediction is a remarkably inefficient route to platonic representations.

Two showcases follow. One rotation turns separately trained encoders into an image captioner, scoring a CIDEr of point eight three eight on COCO. And even that rotation needs no paired data. On the latest pair-free benchmark, the text JEPA beats every zero-shot accuracy reported with language models of up to eight billion parameters. On ImageNet-100, for example, it reaches sixty-four percent top-5 accuracy, against at most fifty-six percent.

Everything turns on the condition in the theorem. The guarantee of the same representation up to one rotation applies when positive pairs are good views, two observations of one scene. For other views the abstract claims only that misalignment is bounded, not that it vanishes. And the claim about language models rests on treating next-token prediction as a JEPA whose views are parts of a single document, which is exactly the case that falls outside the guarantee.

The conclusion the authors draw is a shift in where to look. Scale alone has been the explanation for why representations converge. Here the lever is the choice of positive views. Pick pairs that genuinely show the same scene from two sides, and convergence is a theorem, reached with far fewer samples and far smaller models than scale would demand.