PhysOPSD: Code as Privilege for Physically Plausible Video Generation

PhysOPSD. Code as Privilege for Physically Plausible Video Generation. By Harold Haodong Chen and five others, from the Hong Kong University of Science and Technology and Alibaba Group. Published on the ninth of November.

Video generative models now reach remarkable visual fidelity, yet they stay brittle once a clip needs physically plausible dynamics or complex interactions. Targeted post-training is the obvious lever, and both usual forms have a weakness. Supervised fine-tuning suffers from offline state mismatch. Reinforcement learning is held back by physical reward functions that are sparse and poorly defined. On-policy self-distillation, called OPSD, reconciles on-policy rollouts with dense corrective feedback, which moves the real difficulty somewhere else: designing a good privileged context. Familiar choices fall short. A text prompt is inherently declarative. A real reference video tangles physical law together with visual appearance, and offers no parametric control over either.

PhysOPSD makes executable code the privileged physical context. A static visual observation can only show a world. Code instantiates one, an editable physical world with abstraction and controllability built in. Where text only declares what should happen, and footage mixes the physics with how things look, a program states the physics on its own terms, and its parameters can be changed. That program fills the slot on-policy self-distillation left open: the privileged context the whole approach now hinges on.

A component called PhysOPSD-Engine translates real-world physical scenarios into verifiable simulation programs. On top of that, PhysOPSD adds two ingredients. The first is dual-level physical privilege, which bridges the information gap through an intermediate bottleneck. The second is a set of controlled physical variations, generated under shared rollout noise, so that the causal shifts in dynamics are exposed rather than hidden among unrelated differences. Holding the rollout noise fixed while the physics varies means that what differs between two rollouts can be traced to the physical change itself.

Across diverse backbones, extensive experiments show PhysOPSD substantially improving physical plausibility. It outperforms both supervised fine-tuning and reinforcement learning baselines while using only three thousand samples, and it does so while preserving general video generation quality. The gain in physics, in other words, is not bought by giving up the visual fidelity these models already had.

The argument runs in a chain. Distillation solves the feedback problem, which leaves the context problem. Text says too little, real footage says too much at once, and code sits between them: a description of physics that can be run, checked, and changed one parameter at a time.