Agentic Visual Generation: From Generative Models to Agentic Control

Agentic Visual Generation. From Generative Models to Agentic Control. By Yinming Huang and nine others, from Fudan University, the Shanghai Innovative Institute, and three other institutions. Published on the sixth of September. Its code repository is public but slight: one source file.

Visual generation is changing shape. It started with generative models called once, prompt in and image out. It is becoming agentic control: processes that plan, select tools, inspect the outputs they synthesise along the way, revise failures, and reuse what they learned before. In most such systems, the controller is an LLM or a VLM, and the visual generation models serve as its tools or executors. The intelligence that decides sits in the language model, and the generator carries out what it is told.

What the field lacks is a consistent criterion for when a generation system becomes agentic at all. Planning depth, tool use, collaboration among multiple roles, and reinforcement learning are all commonly treated as evidence of agenticity. Yet none of them necessarily determines which generation decisions the controller is actually able to make. A system can plan deeply and still have no say over what gets generated. Without a shared criterion, calling one system more agentic than another is hard to pin down, because different papers are pointing at different properties.

This survey organises the field by a single question: what can the controller directly control in the generation process? The answer defines levels. At level one, Conditioning Control, the controller prepares the input to a predetermined generator, but does not control which visual operation is executed. It shapes the request, but the operation that follows is fixed in advance. At level two, Execution Control, it selects and invokes the operations themselves: generation, editing, rendering, or other operations that modify content. Here the controller is deciding what actually happens to the image, not only what the generator is asked.

At level three, Outcome-Adaptive Control, it observes an intermediate outcome and uses that observation to change a later operation within the current task. At level four, Experience-Adaptive Control, it keeps experience from completed tasks and uses it to change decisions on future tasks. The difference between these two is the time span: level three adapts within one task, and level four carries what it learned from one task into the next. A separate level zero, Fixed Support, covers generators, editors, evaluators, reward models, benchmarks, and fixed pipelines that have no deployed controller making generation-level decisions.

The levels describe a progressively broader scope of decision-making, and they deliberately do not track other things. Not model size, not system complexity, not output quality, not the number of tools or roles, and not the training method. Each step up adds a kind of decision: first over inputs, then over operations, then over reactions within a task, then over learning across tasks.

The framework is applied across a wide span of generation: image, video, editing, 3D, world, slide, and user-interface generation. Read through it, the field shows how controller capabilities have evolved, and how the mechanisms that deliver them are distributed across the levels. The same scale works whether the output is a single picture, a video, an edited image, a 3D scene, or a slide.

The value of the scheme lies in what it refuses to count. By setting aside planning depth, tool count and training method, and asking only which decisions the controller holds, it gives a way to compare systems that otherwise look alike, or unlike, for reasons that have nothing to do with control.