AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

AV-GRPO. Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation. By Zhiyu Xu and six others, from the Shanghai AI Laboratory, the Hong Kong Polytechnic University, and three other institutions. Published on the twenty-fourth of September. Its code repository is worked out: about six hundred source files, with a readme.

Joint audio-video generation has made major progress, and its models still fall short in three places: the fidelity of each modality on its own, how well the output matches the text, and how tightly sound and picture are synchronised. Reinforcement learning after pre-training looks like a remedy, but adapting it directly to this setting runs into three obstacles. Rewards from different modalities are heterogeneous, which entangles the learning signals and makes credit hard to assign. Optimising the two modality towers jointly is computationally expensive, because their dynamics diverge. And how hard synchronisation is to judge depends on the particular paired sample, which keeps reward comparisons from being fair.

AV-GRPO is a modality-anchored online diffusion reinforcement learning framework, released together with 5DAV, a training dataset that is decoupled and controllable in difficulty. The central move is a conversion. A coupled problem, learning preferences over two modalities at once, is turned into unimodal subproblems, so that each reward can be attributed precisely and synchronisation improves as a result.

Three modules make that conversion. Modality-anchored rollouts disentangle the learning signals and stabilise difficulty. Trajectory-locked optimisation with a frozen tower reduces cost and reassigns credit. And the objectives and perturbation strengths are adaptive, tailored to the dynamics of each modality. The dataset plays its part too: 5DAV decouples its samples across five dimensions so that training can be systematic.

Experiments on JavisBench and VABench show AV-GRPO outperforming LTX-2.3 on all three of the original complaints: generation quality, semantic alignment, and cross-modal synchronisation. That holds under LoRA and under full fine-tuning. Ablations confirm the individual design choices.

Almost every piece of this work pulls audio and video apart. Where the existing approach would optimise audio and video as one tangled objective, AV-GRPO anchors on one modality at a time, holds the other tower still, and tunes its objectives to the modality in hand. Even the data is decoupled. The bet is that synchronisation, the property that needs both modalities, is best improved by first pulling them apart.