LoopVL: Recurrent Visual Intelligence
Zhe Qian, Ziyang Gong, Zhongxing Xu, Hehan Li, Zhonghua Wang, Fei Luo, Mingxuan Wang, Xue Yang and 4 others
Renmin University of China · Baidu · Shanghai Jiao Tong University · Monash University · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems
Published September 29, 2026 · on the feed October 1, 2026 · ▲ 454 · arXiv 2609.38426
The abstract
We introduce LoopVL to study whether Loop Transformers can be effectively extended to vision- language models. LoopVL combines Module-Loop and Model-Loop computation to iteratively update a unified vision-language state through shared modules. We train LoopVL from scratch through language pre-training, multimodal training, and post-training. LoopVL outperforms a range of similarly sized and larger non-recurrent models on multimodal understanding and visual reasoning benchmarks. We also observe Visual Aha Moments in LoopVL, characterized by pronounced shifts in visual attention across loops. LoopVL provides practical evidence for recurrent vision-language modeling and offers an intuitive perspective on how shared parameters can support deeper multimodal computation over continuously evolving visual-language states.
The Brief
1:42A short podcast summary of the paper. Written and voiced by AI.
Transcript
LoopVL. Recurrent Visual Intelligence. By Zhe Qian and eleven others, from Renmin University of China, Baidu, and six other institutions. Published on the twenty-ninth of September. Its code repository is worked out: about ninety-five source files, with a readme and a licence.
The starting point is the Loop Transformer. The question the authors pose is whether Loop Transformers can be extended effectively to vision-language models, which have to work over what they see and what they read at the same time. The abstract frames it as an open question to be studied rather than an assumption.
LoopVL is their answer. Its central object is a single, unified vision-language state, and the model updates that state iteratively through shared modules. Rather than passing once through a stack of distinct layers, the state is revisited and refined, loop after loop.
Two kinds of looping are combined to do this, named Module-Loop and Model-Loop computation. The abstract names them without detailing how each is built, so the point to hold onto is that both serve the same iterative update. The model is trained from scratch, in three stages: language pre-training first, then multimodal training, then post-training.
On benchmarks for multimodal understanding and for visual reasoning, LoopVL outperforms a range of non-recurrent models, both those of similar size and larger ones. The authors also report something they call Visual Aha Moments: pronounced shifts in where the model's visual attention falls, from one loop to the next.
LoopVL is offered as practical evidence that recurrent vision-language modeling works. Beyond the scores, it gives an intuitive way to picture what shared parameters buy: deeper multimodal computation over a visual and language state that keeps evolving, with the attention shifts as one visible trace of that evolution.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Depth usually costs parameters. This quest asks whether a vision-language model can get it another way.
You want a small vision-language model to reason more deeply about images. The usual way to get more depth is to stack more layers, each with its own parameters, and the model grows with every one.
The road ahead
- 1 · The problem ~6 min Small vision-language models get deeper only by growing bigger, and reusing layers has been studied largely in language, not once images join the text.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min LoopVL reruns shared modules over one joint image-and-text state, adding depth without parameters, and its visual attention shifts between passes.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min LoopVL is trained from scratch in three stages: language pre-training, then multimodal training, then post-training.
- 6 · The experiment ~6 min You’ll guess: How did LoopVL compare with non-recurrent models, both of similar size and larger, on multimodal understanding and visual reasoning benchmarks?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.