ProAR: Learning Prospective Reasoning with Autoregressive Video Models

ProAR. Learning Prospective Reasoning with Autoregressive Video Models. By Linghui Shen and three others, from the Hong Kong Polytechnic University, the University of California, Davis, and one other institution. Published on the second of October. Its code repository is worked out: about sixty-five source files, with a readme and a licence.

Autoregressive video models are good at causal generation, producing a video one chunk after another. That same reliance on predicting the next chunk confines them to a short-sighted, reactive way of working: each step answers the last one, and nothing looks ahead. For ordinary generation this may be tolerable. For reasoning-oriented generation it is a real limitation, because there what matters is reaching a target outcome through valid intermediate states, not just making each local stretch of video look plausible.

ProAR turns autoregressive video generation into a goal-oriented reasoning process. Prospective, in the title, is the opposite of reactive: a model that generates with the outcome already in view, rather than discovering it one chunk at a time. ProAR does this with two components that work at two different ranges. One anchors generation to the long-range outcome, so the model knows where the video is supposed to end up. The other guides the short-range transitions from one step to the next, so that each step is taken with some anticipation of the ones that follow. Neither replaces the autoregressive loop. Both are added to it.

For the long range, goal-frame prediction is brought inside the autoregressive loop. An asymmetric attention mask lets the predicted goal frame guide the generation of intermediate states, while preventing those intermediate states from disrupting the goal frame in return. The goal steers the path, and the path cannot rewrite the goal.

For the short range, the authors introduce future representation self-alignment, which encourages the current hidden states to anticipate the temporal dynamics that are coming. Because autoregressive training uses teacher forcing, clean representations of the future are available in a single forward pass. Current representations are aligned with them through a lightweight predictor that is used only during training. Put together, the two mechanisms combine explicit, sparse supervision on the target with implicit, dense guidance at every step, at modest computational cost.

Experiments show the complementary components consistently improving performance across diverse visual reasoning benchmarks, which is the evidence that the long-range and short-range pieces each contribute rather than one doing all the work. Training is efficient too: ProAR surpasses standard autoregressive baselines trained to completion while using only twenty-five percent of the training steps. Put the other way round, a quarter of the training budget is enough to overtake the full one. The paradigm also shows promising applicability to embodied reasoning tasks, though the abstract gives no figures for those.

What ProAR adds is foresight at two scales. A model that only ever predicts its next chunk is given a destination to aim at and a habit of anticipating what comes next, and neither requires giving up the causal, chunk by chunk generation it started with.