Turns Matter: Advancing Streaming Video RL via Turn-Level Credit
Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao, Hongrui Wu, Haowen Hou, Yongfu Zhu, Xuehui Wang and 15 others
Fudan University · Shanghai Innovation Institute · JD.com · Stanford University · Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong
Published November 4, 2026 · on the feed November 5, 2026 · ▲ 51 · arXiv 2611.06728
The abstract
Streaming video understanding requires models to repeatedly decide whether to respond or continue observing. Recently, Group Relative Policy Optimization (GRPO) has been applied to this setting, with existing approaches emphasizing reward design while retaining a single trajectory-level advantage across turns. Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures. We propose TurnWise, a turn-level GRPO framework that aligns learning with individual streaming decisions. TurnWise addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy. Specifically, turn-level credit assignment uses silence-completed prefix scoring to construct group-relative advantages from each decision's remaining return, separating its learning signal from earlier rewards. Turn-mean policy optimization averages token losses within each turn and then across turns, giving response and waiting decisions equal weight regardless of output length. These complementary designs improve streaming policy learning without a learned critic or additional rollouts. Extensive experiments show our method outperforms prior RL approaches for streaming video understanding.
The Brief
1:49A short podcast summary of the paper. Written and voiced by AI.
Transcript
Turns Matter. Advancing Streaming Video RL via Turn-Level Credit. By Shuai Dong and twenty-two others, from Fudan University, the Shanghai Innovation Institute, and eight other institutions. Published on the fourth of November.
Streaming video understanding asks a model to decide, again and again, whether to respond now or keep observing. Group Relative Policy Optimization, or GRPO, has recently been applied to that setting. Existing approaches concentrate on reward design, while retaining a single advantage for the whole trajectory, shared across every turn. Shared credit misfires in two directions. After an earlier success, it can reinforce redundant replies that follow. After an earlier failure, it can discourage follow-ups that would have been useful.
TurnWise is a turn-level GRPO framework that aligns learning with the individual streaming decisions. It targets the mismatch at the root of the problem: feedback arrives at the level of the trajectory, while the interaction happens turn by turn. The intended effect is to promote informative responses and reduce redundancy.
Two designs carry it. The first is turn-level credit assignment. Using what the authors call silence-completed prefix scoring, it constructs group-relative advantages from each decision's remaining return, which separates that decision's learning signal from rewards earned earlier. The second is turn-mean policy optimization. Token losses are averaged within each turn and then across turns, so a decision to respond and a decision to wait carry equal weight, regardless of how long the output is.
The two designs are complementary, and they improve streaming policy learning without a learned critic and without additional rollouts. Extensive experiments show the method outperforming prior RL approaches for streaming video understanding.
Seen from a distance, TurnWise leaves the question of reward to others and changes who gets credit for it. Each decision is judged on what came after it, and each turn counts once, whether it produced a long answer or a short wait.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model is watching a live video and must keep choosing: speak now, or wait. One shared score is teaching it the wrong lessons.
You are training a model that watches a live video and, at every turn, decides whether to reply now or keep watching. Reinforcement learning scores each whole episode, and every turn in it shares that one score.
The road ahead
- 1 · The problem ~7 min When one episode score is shared by every turn, later turns get praised or blamed for what earlier turns did.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Credit each turn by what it and later turns added, measured against a silent-from-here version, and weigh every turn equally in the loss.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Score silence-completed prefixes, subtract to get remaining returns, compare them in the group, then average losses per turn and across turns.
- 6 · The experiment ~6 min You’ll guess: To give each streaming decision its own credit, what extra machinery did TurnWise need?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.