Less from More: Reinforcing Sparse Video Reasoning from Dense References

Less from More. Reinforcing Sparse Video Reasoning from Dense References. By Wenfang Sun and two others, from the University of Amsterdam. Published on the seventh of October. Its code repository is worked out: about eighty-five source files, with a readme and a licence.

Video-language models commonly assume that more temporal observations lead to more reliable reasoning. The authors question that assumption. In their view the key challenge is not merely processing more video frames efficiently, but learning to reason reliably under limited temporal evidence: when the model is given only a sparse handful of frames to work from. Seen that way, efficiency and reliability are different problems. Handling more frames faster does not teach a model what to do when most of the video is simply not there.

SAVER is a dense-to-sparse post-training framework. At inference, the model works from sparse frames. During training, dense views of the same video serve as references. The dense view exists only at training time, as a guide to what the sparse view should learn to hold on to.

Training is reinforcement post-training on paired dense and sparse views. The pair is optimized with grounding rewards and with a reliability-gated reference reward, and together these encourage the sparse view's predictions to preserve task-relevant temporal evidence. The data budget is small. SAVER is trained only on one thousand two hundred and fifty randomly sampled temporal grounding examples, and no video question answering annotations are used at all.

Evaluation covers three temporal grounding benchmarks and six video question-answering benchmarks, and across them SAVER consistently improves performance across frame budgets. In particular, it can match or surpass dense-frame Qwen3.5 baselines while using substantially fewer frames.

A second lesson sits beside the frame savings. Training touched only temporal grounding, yet the gains reached video question answering. The authors take this to show that temporal grounding can act as an effective evidence-localization proxy: a way of learning sparse video reasoning that transfers to broader video understanding tasks. The title compresses the whole argument. Less at inference, learned from more at training, with the dense view acting as a teacher that the sparse view eventually does without.