PhyRubric: Rubric-Grounded Diagnostic Evaluation of Physical Commonsense in Text-to-Video Generation

PhyRubric. Rubric-Grounded Diagnostic Evaluation of Physical Commonsense in Text-to-Video Generation. By Huiyi Chen and seven others, from the University of Illinois Chicago. Published on the tenth of November.

Text-to-video models are increasingly positioned as world simulators, which makes it essential to ask whether what they generate conforms to physical commonsense. Existing benchmarks ask that question by having an annotator watch a video and rate it with a single score, based on intuition. Two things go wrong. Resting on intuition, the rating is subjective, so two annotators need not agree and neither of them has to say why. And being a single number, it is silent about which physical principle the video actually broke. For a model being sold as a simulator of the world, that silence is the expensive part: you learn that something looked wrong without learning what.

PhyRubric replaces the single intuitive score with a rubric, and the order of operations matters. For each caption, a caption-specific rubric is constructed before any video has been seen, decomposing that caption into semantic items and physical items. Scores are then derived from verdicts on those items one at a time, and each verdict is grounded in timestamped evidence: a point in the video where the item was met or was not. Writing the rubric before the video exists is the part that does the work, since the standard cannot be bent to fit whatever the generator happened to produce. The output is diagnostic rather than a number standing on its own.

The framework is instantiated as PhyRubric-Bench, spanning thirty-four categories across five physical domains. It is then scaled up into PhyRubric-Train, three thousand seven hundred and eighty-eight videos, and that set is used to fine-tune an open-source judge, PhyRubric-Judge.

Ten text-to-video models are run through it, and the picture is that physical commonsense remains far from solved. On the judging side, PhyRubric-Judge correlates more closely with human judgments than existing open-source judges do, while matching the frontier proprietary model Gemini 4 Argon. The item-level analysis then feeds back actionable insights for the generators themselves.

Decomposing a caption before the video arrives is what changes the character of the evaluation. A score can still be produced, but it now comes with the particular principle that was violated and the moment it was violated, attached.