Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
Zhefan Rao, Meng Chu, Xuan Zhang, Jingfan Chen, Guohao Cai, Runyi Li, Shiyi Du, Ning Lu and 75 others
The Hong Kong University of Science and Technology · National University of Singapore · The Hong Kong Polytechnic University · The University of Tokyo · Carnegie Mellon University · University of California, Berkeley
Published October 22, 2026 · on the feed October 26, 2026 · ▲ 36 · arXiv 2610.31066
The abstract
World Models (WMs) can support Multimodal Agents (MAs) through simulated experience, while Agent trajectories can inform World Model refinement. Evaluating this interaction requires separate evidence about action selection and state evolution. We introduce Dual Exam, which tests Agents under fixed environment response mechanisms and World Models under fixed action interventions across physical, digital, social, and scientific domains. The Agent track compares 10 configurations on 1,088 items in 15 reporting categories. The World Model track contains 400 cases across eight scenarios; 15 configurations have full coverage and one covers six scenarios. GPT 6 Astra leads the case-weighted Agent Index at 31.3, and GPT-Image-2 leads the full-coverage WM Overall at 21.7. These scores use different tasks and metrics and are not directly comparable. A 150-case pilot of one coupled pairing records zero task successes and motivates component-level diagnosis. Results reveal uneven task performance and failures to preserve task-relevant state. Their value for predicting coupled success or sustained reciprocal learning remains to be established.
The Brief
1:59A short podcast summary of the paper. Written and voiced by AI.
Transcript
Dual Exam. Controlled Evaluation of Multimodal Agents and World Models. By Zhefan Rao and eighty-two others, from the Hong Kong University of Science and Technology, the National University of Singapore, and twenty-five other institutions. Published on the twenty-second of October. No code repository is linked to it.
World models and multimodal agents can feed each other. A world model can support an agent through simulated experience, and an agent's trajectories can inform how a world model is refined. But when the two are run together, a failure could come from either side. Evaluating the pairing needs separate evidence about two things: how actions are chosen, and how the state evolves.
Dual Exam supplies that separation by holding one side fixed at a time. Agents are tested under fixed environment response mechanisms, so what is measured is action selection. World models are tested under fixed action interventions, so what is measured is how the state evolves. Both tracks span physical, digital, social and scientific domains.
The Agent track compares ten configurations on one thousand and eighty-eight items, in fifteen reporting categories. The World Model track has four hundred cases across eight scenarios. Fifteen configurations cover all of them, and one more covers six of the eight.
GPT 6 Astra leads the case-weighted Agent Index, at thirty-one point three. GPT-Image-2 leads the World Model Overall among full-coverage entries, at twenty-one point seven. The authors stress that the two tracks use different tasks and metrics, so these scores are not directly comparable. Across both, results show uneven task performance and failures to preserve task-relevant state.
There is also a coupled test. A one-hundred-and-fifty-case pilot of one agent paired with one world model recorded zero task successes, which is what motivates diagnosing each component on its own.
Run as a coupled pair, an agent and a world model gave zero successes and no indication of which part broke. Examined apart, each now has its own score, though whether those scores predict coupled success, or sustained learning between the two, is left to be established.
Paper Quest · 7 chapters · about 37 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Two models, one task, zero successes. The question is whose fault it was.
You pair an AI that chooses actions from images and screens with a model that simulates what happens next. The hope is that each helps the other improve.
The road ahead
- 1 · The problem ~7 min When an action-choosing agent and a simulator of what happens next fail together, the joint result alone cannot tell you which one went wrong.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Dual Exam tests each component alone, holding the other side fixed, and scores recorded evidence against the task's goals.
- 4 · Choose your own research ~3 min
- 5 · How it works ~3 min
- 6 · The experiment ~7 min You’ll guess: In a pilot pairing one agent with one world model on coupled tasks, how often did the pair succeed?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.