DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation

DramaChain Bench. An End-to-End Benchmark for Short-Drama Generation. By Haoyuan Shi and fourteen others, from Tencent Hunyuan, the Beijing Film Academy, and two other institutions. Published on the first of September. No code repository is linked to it.

Commercial short dramas are made in a chain of stages: a script, then a storyboard, then keyframe images, then video shot by shot, and finally the finished drama. Most existing benchmarks evaluate only the video-generation stage, and they feed it inputs written in advance rather than what real upstream stages produce. Two questions are left without answers. Does each stage stay faithful to the original script's intent, and not merely to the prompt directly in front of it? And do separate shots still hold together once they are assembled into releases running several episodes?

DramaChain Bench is presented as the first short-drama benchmark to evaluate every stage of the full production chain. All of it shares one dimension system, DramaChain Dimensions: five evaluation axes, applied at every stage, which resolve into sixty-three leaf dimensions. Using the same axes from the script through to the finished episode is what makes it possible to ask whether a later stage has kept faith with an earlier one, rather than judging each stage in isolation.

Three in-house systems sit on top of that. DramaChain Agent is calibrated against commercial short-drama platforms, both in workflow and in the quality of the finished drama, so that models can be compared fairly stage by stage. The DramaChain Labeling System has three professional annotators score each of five thousand seven hundred and eighty-five items independently. Every defect is localised in space and time and chosen from a predefined defect list.

Because each defect is located and named, a score can be traced back to what caused it. That process yields seventeen thousand four hundred and eighty-eight valid scores. It also yields two hundred and fifty-five thousand nine hundred and twenty-five attribution records, each traceable. Finally, DramaChain Agentic Judge scores every leaf dimension automatically. It gathers evidence over several agentic rounds, then judges against a checklist specific to each item.

The human annotations show that defects introduced upstream cascade down the pipeline. Final episode quality, in other words, is not governed by video generation alone, which is exactly what a benchmark of the video stage by itself would miss. The automatic judge reproduces the human model ranking with a mean PLCC of zero point nine one eight. The authors take that as enough to admit new models without paying for new annotation. The expensive human labeling, in that reading, is done once to establish ground truth, and the judge carries the benchmark forward from there.

The argument for measuring the whole chain rests on that cascade. If errors in the script or the storyboard surface later as bad episodes, then grading only the last step blames the wrong stage, and an end-to-end benchmark is the only kind that can see where a failure began.