Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports

Beyond Watching and Searching. Benchmarking Agents for Video-centered Deep Research Reports. By Jiahao Meng and six others, from Peking University, Alibaba Group, and one other institution. Published on the twenty-second of October. Its code repository is public and real: ten source files.

Deep research agents gather evidence from the open web and synthesize it into long reports. Current benchmarks for them, though, do not evaluate research that is centered on a source video. Watching a video and searching the web are both covered somewhere. Research that starts from a video and reaches outward is not. An agent asked to research what a video shows has to understand the video, go find evidence beyond it, and tie what it finds back to the video itself. Current benchmarks do not measure that combination.

The authors define the task as Video-Centered Deep Research Report Generation. Given a video and a user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to the relevant moments in the video. The practical value is direct: such a report helps users locate relevant moments, verify claims, reproduce procedures the video demonstrates, and make informed decisions from what it contains. Alongside the benchmark, OmniDR-Bench, they present a skill-based workflow for the task.

Evaluation runs on Omni Atomic Rubric Trees, or Omni-ARTs. Each tree has three dimensions, twenty-three shared directions, and then atomic binary criteria specific to each task. The directions test audiovisual grounding, the use of multimodal evidence, and interactive presentation. The trees are built by a process called Audited Rubric Construction, or ARC, which combines independent evidence collection, automated checks, behavioral calibration, and iterative human review.

OmniDR-Bench contains fifty tasks, in Chinese and English, spread across nineteen real-world subcategories. Together they carry six thousand four hundred and fourteen criteria.

Eighteen agent configurations were tested. The best system scores sixty-three point three out of a possible one hundred. The authors name three places where current agents fall short. They need broader coverage of the content, stronger causal and logical analysis across modalities, and more accurate support for their citations. The analysis further breaks results down by models, harnesses, skill configurations, video categories, and research trajectories.

The design choice that carries the benchmark is the rubric tree: thousands of small yes or no criteria, organised under shared directions and audited by humans, so that a long interactive report can be scored piece by piece. Against that standard, the strongest of eighteen configurations still falls short of full marks, and the shortfalls are named: coverage, reasoning across modalities, and citations. The code and data are released on GitHub.