Principia: Relational Physics Tests for Video Models

Principia. Relational Physics Tests for Video Models. By Varun Varma Thozhiyoor and three others, from the Indian Institute of Science and Johns Hopkins University. Published on the third of September. No code repository is linked to it.

Evaluating physical reasoning in video models runs into a measurement problem before it gets anywhere near physics. Absolute motion measurements depend on frame rate, on object scale, and on camera calibration. In generated video, all three are frequently ambiguous or simply unavailable. You cannot ask whether a falling object accelerated correctly if you do not know how big it is, how far away it is, or how fast the clock is running.

The way around that is to stop measuring absolutes. When two objects in the same scene obey the same physical law, their motions have to satisfy predictable relationships with each other, and those relationships hold whatever the calibration happens to be. Two balls dropped together fall together, and that stays true whether or not you know the frame rate. Principia is built on exactly this: a benchmark that evaluates Newtonian physics through relational consistency between paired objects.

It spans eight phenomena. Gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation, covering translational, rotational, collisional and oscillatory dynamics. The scenes are real, recorded under controlled protocols rather than simulated. Alongside the tests comes a calibration-independent consistency score that quantifies physical violation directly in image space, which is the part that makes the whole thing usable on video whose provenance you do not control.

Across thousands of generations from six state-of-the-art video generators, no model exceeds zero point four two on Principia. The same models all score around zero point eight on VBench. Vision-language models were also tested, on whether they could detect relational physics violations rather than produce them, and the best managed only sixty-seven percent accuracy, with most performing near chance.

The gap between zero point eight and zero point four two is the finding. Video that looks right and video that moves right are being measured by different instruments, and only one of them was being reported.

Principia. Relational Physics Tests for Video Models. By Varun Varma Thozhiyoor and three others, from the Indian Institute of Science and Johns Hopkins University. Published on the third of September. This is the long version: the whole paper rather than the abstract.

Video generators are increasingly described as world models, meaning systems you could use to predict the consequences of an action. That is a much stronger claim than making convincing video. This paper builds the instrument that separates the two, and the headline result is that the gap is enormous. Across six state-of-the-art generators, none exceeds roughly zero point four on Principia, while all of them score around zero point eight on VBench, the visual quality benchmark the field has been optimizing against.

The obstacle to measuring physics in generated video is that physics is usually measured in absolute quantities, and generated video does not come with any. You want to ask whether a falling object accelerated correctly, and to answer that you need the frame rate, the object's scale, and the camera calibration. In generated video all three are typically ambiguous or unavailable. Earlier attempts each work around this differently and each pays for it. Some judge whether motion looks plausible, but plausibility is subjective and a visually convincing clip can be physically wrong. Some compare a generated trajectory against a recorded one, which penalizes any deviation from a single reference even when many futures are physically valid. Some evaluate against explicit physical laws, which drags the metric quantities back in. And single-phenomenon studies, free fall under gravity being the common one, reveal a targeted failure without telling you whether a model preserves physical structure generally.

That is the whole benchmark. For each phenomenon the authors specify the relation two paired objects must satisfy, then measure how far the generated motion departs from it, aggregating those deviations into a continuous invariance score computed purely in image space. Physical laws impose several kinds of invariant, so the tests are not all equalities: some are ratios, some are relative orderings in time or space. Principia spans eight phenomena, chosen to cover four kinds of dynamics. Gravity in free fall, restitution measured during and after collisions, sliding under friction, rotational inertia, momentum, projectile motion, pendulum swinging, and mass-spring oscillation. That covers translational, rotational, collisional and oscillatory behaviour.

Wan2.2-14B leads at an average of zero point four one nine, narrowly ahead of Omni at zero point four zero nine and Veo-3.1 at zero point three eight. The open-weights model beating the closed frontier ones is itself a finding. Two patterns underneath that matter more than the ranking. Scaling within a family does not uniformly help: going from Cosmos-2.5 at 2B to 14B improves friction, inertia and pendulum while making restitution, gravity and momentum worse. Scaling Wan2.2 from 5B to 14B gains substantially almost everywhere, and still loses a little on momentum. And compute does not predict fidelity either. Cosmos-2.5-14B spends more compute per video than Wan2.2-14B, seventy-four minutes against sixty-six, and scores zero point two two lower. Every model does badly on momentum, which is the task involving complex multi-object interaction. The same tests were then put to vision-language models, asking them only to detect violations rather than avoid producing them. The best managed sixty-seven percent accuracy and most performed near chance, so the models we would reach for to check this work are not better at noticing the problem than the generators are at causing it.

Principia tests macroscopic Newtonian mechanics. It says nothing about fluid dynamics, soft-body deformation, or thermodynamics. The authors also note that generators sometimes produce hallucinated objects, missing objects, or severe deformations, failures which are real but unrelated to what this instrument measures.

The sentence to keep is the authors' own. Current generators produce highly realistic textures and visually appealing renderings, but fail to enforce the relational invariants that physical laws require. They have learned to render the world without learning what holds it together. And the reason nobody noticed is that the benchmark everyone was reporting measures the first thing and not the second.