VeriFine: Scaling Verification for Self-Improvement in Embodied Reasoning

VeriFine. Scaling Verification for Self-Improvement in Embodied Reasoning. By Zewei Zhou and twelve others, from NVIDIA, UCLA, and two other institutions. Published on the sixth of October. No code repository is linked to it.

A policy that keeps improving keeps exposing new kinds of failure, and that changes what its judge needs to be able to verify. A fixed judge cannot keep up. It constrains the feedback available for optimization, and it constrains the discovery of useful training examples, so self-improvement stalls. The difficulty is sharper in embodied reasoning, where a reliable evaluation has to account for spatial grounding, causal reasoning, and decision-making that is aware of safety.

VeriFine is an agent harness framework that scales verification by letting three things evolve together: the policy, the training curriculum, and the judge. Rather than holding the judge still while the policy moves, the judge is improved too, at the moments when it becomes the limit.

The work is split into two loops. In the Policy Improvement Loop, a rubric judge diagnoses recurring failures, builds an adaptive curriculum from them, and the policy is optimized against it. When progress plateaus and verification turns into the bottleneck, the Judge Improvement Loop takes over. It selectively asks for human guidance on failure cases that are informative, and refines the judge through what the authors call coactive calibration. Humans and agents resolve their disagreements and converge toward the objective rubric of physical reasoning. The revised judge then steers the next round of data selection and policy optimization, and the cycle continues.

Experiments cover driving and robot navigation tasks, two settings where spatial grounding, causal reasoning and safety all bear on whether a decision is right. The results show continuous self-improvement on both sides of the system: the policy gets better, and so does the judge's capability. That pattern holds across two training regimes, reinforcement learning and supervised fine-tuning, which suggests the loop is not tied to one way of optimizing the policy. The abstract reports this as a trend of sustained improvement rather than through headline numbers.

The argument is that verification is not a fixed piece of infrastructure but something that has to grow alongside the policy it checks. As failure patterns shift, the judge is brought forward with targeted human help, and that is what lets improvement keep going instead of levelling off.