Reviso: Not All Visual Evidence Deserves Equal Attention in Video Captioning

Reviso. Not All Visual Evidence Deserves Equal Attention in Video Captioning. By Tengfei Liu and fourteen others, from Peking University, KlingAI Research, and six other institutions. Published on the second of November.

Multimodal large language models can already write detailed captions for videos. The trouble is where the detail goes. They may dwell on minor details and miss the essential context and the key events. The authors' argument is that visual evidence does not all deserve the same attention. What matters in a video depends on what that particular video contains. A cooking clip and a sports highlight call for attention in different places, and a captioner that spreads its effort evenly will spend too much of it in the wrong ones.

Reviso is an agentic video captioning framework built on that argument. It learns two things: what to observe, and how to gather the visual evidence that is relevant. Instead of treating every frame alike, it plans its looking and then goes back to look. The plan comes first, the closer inspection follows it, and both the plan and the way of following it are learned rather than hand-written.

It starts from a global preview at a low frame rate. From that preview, Reviso builds a hierarchical observation plan called Context, Focus, Points, or CFP. The plan names the context of the video, ranks the key aspects, and sets concrete observation targets along with the tools for each. Guided by the plan, Reviso does goal-directed active perception. It selectively revisits temporal segments and combines tools for fine-grained inspection. It interprets what it finds, updates the plan when needed, and finally merges global and local information into the caption.

Learning that strategy takes two stages. First, a verification-guided pipeline builds high-quality training trajectories, using factual verification and teacher-guided refinement. Then Reviso is trained with Group Relative Policy Optimization, or GRPO, using three signals. A key-fact reward targets caption quality. A Plan Alignment Reward, PAR, rewards picking out the important observation targets. And an Unproductive Tool-Call Penalty, UTCP, adjusts the penalty for observation cost according to how good the caption turns out. Taken together, the rewards pull in two directions at once: toward captions that get the key facts right and look at the things that matter, and away from tool calls that cost effort without paying off in the caption.

Across four video captioning benchmarks, Reviso outperforms the open-source baselines it was evaluated against. Its captions also do a better job of supporting downstream video question answering. That second result treats a caption as something to be used, not just read: a description that helps answer questions about the video is one that kept hold of what matters in it.

The shift here is from seeing everything evenly to deciding what deserves a closer look. A captioner that plans, revisits and pays for each tool call is being taught to spend its attention the way the title suggests: unequally, and on purpose.