TempCloze: Can Video-LLMs Identify the Missing Middle?

TempCloze. Can Video-LLMs Identify the Missing Middle?. By Wenqi Pei and six others, from the University of Hong Kong, the National University of Singapore, and one other institution. Published on the first of September. Its code repository is public and real: about thirty-five source files.

Benchmarks for temporal reasoning in Video-LLMs usually ask their questions through language. That leaves room for shortcuts that have nothing to do with watching the video. A model might lean on how the options are worded, on correlations among answers, or on plain language priors. A score earned that way says little about whether the model reasons visually about time.

TempCloze is built to reduce those shortcuts. It is a video cloze benchmark, meaning a fill-in-the-blank test for video. A model sees the beginning clip and the ending clip of a video and must identify the true missing middle from four candidates. The answer has to come from reasoning about what happened between the two ends.

The benchmark contains one thousand five hundred and twenty-one carefully filtered videos from seven sources, mostly long-take and egocentric footage. The distractors come from the same source as the real middle, and they vary along three dimensions. Semantic asks what event should happen. Alignment probes when it should occur. Progression tests how it should unfold. Because the candidates share scenes and objects, appearance alone gives little away.

The evaluation covers ten proprietary and twenty-one open-source Video-LLMs, and it points to Alignment as the primary bottleneck. Models often recognize plausible semantic content, and they follow local event progression, but they struggle with temporal alignment: placing the event at the right time.

Two harder variants, TempCloze-Mixed and TempCloze-Hard, are used with four representative models to analyse error patterns and behavioural sensitivity. Those analyses look at where errors arise and how several factors sway the model's choice: the order of the candidates, the direction of the context, how much of the video is visible, frame density, and test-time scaling.

By stripping language out of the answer, the benchmark isolates a narrower question: given two ends of a video, can a model find the middle? Its answer is that knowing what happens, and how it unfolds, comes easier to current models than knowing when.