PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements

PVBench. Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements. By Kaiwei Chen and seven others, from the APRIL Lab at Zhejiang University, the Huazhong University of Science and Technology, and one other institution. Published on the nineteenth of October. Its code repository is worked out: about sixty source files, with a readme and a licence.

Video generation models now produce outputs that look convincing. Whether those outputs are physically plausible is a separate question, and the authors argue it is poorly measured. Physical plausibility here means whether the objects in a generated video obey geometry, kinematics, dynamics and conservation laws. Existing benchmarks fall short in one of two ways. Some rely on coarse judgments from a visual-language model. Others target embodied world models rather than video generation itself. Neither gives a number for whether a generated object moved, held its shape, or conserved what it should have, in the kind of video a text or image prompt produces.

PVBench measures plausibility quantitatively, across forty-six dimensions grouped into seven domains. The domains run from instruction and semantic fidelity, through behavioral and causal consistency, and perceptual and temporal quality. They continue with geometric and spatial consistency, then material properties and field states. The last two are kinematic and spatiotemporal persistence, and mechanical plausibility with conservation laws.

The benchmark has six hundred carefully crafted prompts. Thirty percent of them involve physical embodied agents, such as robot arms, humanoids, quadrupeds and multi-agent systems. The key design choice concerns the dimensions that are hard but physically tractable. For each of those, PVBench pairs the dimension with specialized vision models that produce verifiable, physically grounded measurements.

Those measurements include object-level six-D pose, metric scale, surface normals, kinematics, and the numerical residuals of conservation laws. This is how the benchmark avoids relying only on subjective scoring. It supports both text-to-video and image-to-video generation, and the agreement between its automatic metrics and human judgment is checked through a human evaluation.

With the benchmark in place, the authors evaluate mainstream open-source and closed-source video models and present several insights from the comparison. The abstract does not spell those insights out, so what can be said here is the scope of the evaluation: both kinds of model, measured on the same physically grounded scale, with automatic scores that have been validated against people. That validation matters for reading any result the benchmark produces. A measured pose or a conservation residual is only useful as a score if it tracks what a human observer would call physically wrong, and the human study is the authors' check that it does.

A video can look right and still break the rules of motion. PVBench's answer is to stop asking a model whether a clip looks physical and start measuring the objects in it, where the physics allows a measurement. The benchmark, the prompts and the calibration protocol are being released, with the stated aim of fostering video generation that is physically grounded.