EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL

EmoReward. Self-Evolving Reward Calibration for Emotional Video Generation RL. By Mingzhe Zheng and twenty-one others, from the Hong Kong University of Science and Technology, Tencent Hunyuan, and two other institutions. Published on the fourth of November.

Group Relative Policy Optimization, or GRPO, has advanced the post-training of generative models in verifiable domains such as code and mathematics, where an output can be checked. Video is a harder case. Evaluating video quality, and emotional expression above all, often depends on subjective aesthetics, and there is no uniquely correct target to check against.

The usual workarounds each fall short. Existing approaches typically rely on fixed rewards for visual quality, motion quality and text alignment. Those capture emotional intent poorly, and they can invite reward hacking. Human feedback, in the form of RLHF, is difficult to scale. It is also subject to individual aesthetic differences, which complicate the calibration of facial cues, body motion and colour across different narrative contexts.

EmoReward is a self-evolving training mechanism. Its central move is to learn, from an actual training run, which emotional evidence deserves trust and how much weight it should carry. It distills evidence reliability from the reward traces collected during that run, and uses what it learned to guide how multiple rewards are weighted in subsequent training. The reward is not fixed in advance. It is calibrated by the history of the run before.

Within a run, evaluation proceeds in stages. EmoReward first plans specific grounding cues from the emotional prompt, which then guide the evaluation. During what the authors call the act stage, heterogeneous signal primitives score the generated video. Those scores are not simply added up. Reflective Reward Gating calibrates each primitive's contribution through instance-level gates, one each for phase, face confidence, conflict and subtlety. The weights are then renormalized over whichever primitives remain active, producing the fused reward for that sample.

The self-evolving part happens between runs. Once a training run is finished, RewardMemory estimates each primitive's reliability from rollout scores and outcomes, separately within each emotional regime. From those estimates it fixes bounded multipliers, and the multipliers calibrate the fusion weights in the next training run. The reward models themselves are never updated. What changes is how far each one is believed, regime by regime.

Experiments on HunyuanVideo one point five show improvements across multiple RL algorithms. EmoReward is also compared with other agentic reward methods under SAGE-GRPO. Beyond video, ablations and transfer experiments on SD three point five and on FLUX further support adaptive reward calibration for visual generation in general.

Where a task has no single right answer, EmoReward stops treating the reward as something settled in advance and lets the training record decide which signals have earned trust. Two layers do that work: gates that act on each sample as it is scored, and a memory that carries bounded corrections from one run into the next.