LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents
Zhangxuan Gu, Haoxing Chen, Qi Qin, Yi Xin, Kai Gan, Lin Liu, Long Cui, Xiaomei Wang and 12 others
Inclusion AI · Venus Team · Westlake University
Published September 9, 2026 · on the feed September 15, 2026 · ▲ 16 · arXiv 2609.13287
The abstract
Diffusion large language models (dLLMs) achieve high decoding efficiency through block-parallel, arbitrary-order generation, making them attractive for latency-sensitive applications. GUI agents represent a natural testbed for this paradigm, as they must repeatedly perceive screen states and emit structured, spatially grounded actions in real time. However, whether dLLMs can be extended into capable multimodal GUI agents while preserving their parallel decoding advantage remains an open question. We present LLaDA-UI, a 16.7B-parameter MoE-based, block-wise diffusion vision-language GUI agent. LLaDA-UI follows a two-stage training pipeline: general multimodal pre-training aligns a native-resolution vision encoder with the LLaDA2.0-mini-base diffusion language backbone, followed by GUI-agent supervised fine-tuning on diverse mobile, desktop, web, and grounding data. Across widely adopted grounding benchmarks and navigation benchmarks spanning multiple platforms, LLaDA-UI substantially outperforms Qwen2.5-VL-7B and surpasses Qwen3-VL-8B on four of six reported GUI benchmarks. These results establish block-wise diffusion as a practical generative paradigm for multimodal GUI agents.
The Brief
2:06A short podcast summary of the paper. Written and voiced by AI.
Transcript
LLaDA-UI. Bringing Block-wise Diffusion to Vision-Language GUI Agents. By Zhangxuan Gu and nineteen others, from Inclusion AI, the Venus Team, and one other institution. Published on the ninth of September. Its code repository is public but slight: eight source files.
Diffusion large language models achieve high decoding efficiency through block- parallel, arbitrary-order generation, which makes them attractive wherever latency matters. GUI agents are a natural testbed for that, since they must repeatedly perceive the screen state and emit structured, spatially grounded actions in real time. Whether these models can become capable multimodal GUI agents while keeping their parallel decoding advantage has remained an open question. A GUI agent acts many times in a row, and each action waits on decoding, so decoding speed shapes how responsive the agent can be.
LLaDA-UI is the authors' answer: a block-wise diffusion vision-language GUI agent, MoE-based, with sixteen point seven billion parameters. It keeps the block-wise diffusion approach for generation, and pairs it with vision so that it can read the screen it is acting on.
Training runs in two stages. General multimodal pre-training first aligns a native-resolution vision encoder with the LLaDA2.0-mini-base diffusion language backbone. GUI-agent supervised fine-tuning follows, on diverse mobile, desktop, web and grounding data, so the agent sees the full range of screens it is meant to act on.
The evaluation covers widely adopted grounding benchmarks, and navigation benchmarks that span multiple platforms. Across them, LLaDA-UI substantially outperforms Qwen2.5-VL-7B, and it surpasses Qwen3-VL-8B on four of the six reported GUI benchmarks. The navigation benchmarks span multiple platforms, in line with the mobile, desktop and web data used in fine-tuning.
The authors conclude that block-wise diffusion is a practical generative paradigm for multimodal GUI agents. The parallel decoding is the reason to want the answer. The benchmarks are where the authors test whether it comes at the cost of capability, and on those, the answer they report is that it does not.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
An agent that clicks must also be quick. Start with what slows it down.
You are building an agent that operates phones, desktops and websites. It looks at the screen, emits an action such as a click at coordinates, then looks again, over and over.
The road ahead
- 1 · The problem ~6 min A screen agent decodes a new action at every step, so slow one-token-at-a-time decoding makes it lag, and faster parallel decoders are untested at seeing screens.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Use a block-wise diffusion backbone for fast parallel decoding, align a vision encoder to it, then fine-tune it on GUI data across platforms.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Train in two stages, alignment then GUI fine-tuning; at run time the agent looks at the current screen and writes one structured action.
- 6 · The experiment ~6 min You’ll guess: Against the similarly sized autoregressive Qwen3-VL-8B, on how many of the reported GUI benchmarks did LLaDA-UI come out ahead?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.