World Models' Last Exam in Physics

World Models' Last Exam in Physics. By Mingju Gao and ten others, from Navers Lab at Einsia AI, Peking University, and one other institution. Published on the sixth of October. No code repository is linked to it.

Video world models can produce sequences that look convincing and yet break physics. That raises doubts about relying on them for prediction and planning in embodied AI systems. The existing ways of checking have limits. Many rely on judgments made by another model, or on reference videos to compare against. And the direct physical tests that do exist focus largely on mechanics, leaving the rest of physics mostly untested.

World Models' Last Exam in Physics is a benchmark built on measurement. It evaluates physical consistency in video world models across six areas: mechanics, optics, fluids, thermal and phase-change phenomena, electromagnetism, and surface tension. What it scores are observable physical relationships, tested in a way that can be interpreted, and without any need for a reference video.

There are forty controlled tasks. Each pairs an initial image and a generation prompt with predefined physical criteria, so the model is told where to start and the benchmark knows in advance what physics should follow. The evaluator works in two parts. Task-observability screening comes first, and then task-specific quantitative physical measurements are taken from the generated video. The scores are grounded in measurable evidence, and the benchmark states its measurement limitations explicitly.

Eight video generation models were tested, across one thousand two hundred and eighty videos. Physical inconsistencies persist, and performance varies substantially from task to task. The best model reaches an overall score of fifty-seven point seven six out of a hundred. Two messages sit in that result. The first is that no model tested is close to physically consistent across the board. The second is that an overall score hides a lot: because results swing between tasks, a model can handle one kind of physics and fail another, and the per-task breakdown is where that becomes visible.

The authors also check the checker. On synthetic videos where the physical relationships are known, the measurement module behaves as it should, which gives evidence for its validity under controlled conditions. And compared with asking a vision-language model directly, the evaluator agrees more closely with human judgments, both when ranking within a task and in pairwise comparisons.

The value here is diagnostic. Breadth across physical domains, combined with scores that rest on measurements rather than opinions, gives an interpretable way to see where video world models get physics wrong and to track progress toward ones that get it right. The best of eight models still falls well short of full marks, so there is plenty of exam left to pass.