Wizard: Manipulate Any World in Real Time via Agentic Control

Wizard. Manipulate Any World in Real Time via Agentic Control. By Yuzhe Wu and two others, from Tencent, the National University of Singapore, and one other institution. Published on the twenty-ninth of October.

Video world models now generate virtual environments realistic and interactive enough to open up gameplay that players can customise. Agentic frameworks push that further, by decoupling explicit game logic and controllable world representations from the visual generation itself. Yet the existing approaches support only simple game settings and predefined motion patterns. They generalize poorly to environments and characters they have not seen before, and they offer limited real-time interactivity. Put together, that is a narrow kind of game: a fixed set of moves, in a world the system already knows, with the player waiting on the response.

Wizard is an agentic game world model aimed at all three of those limits. It enables real-time, precise control over the motions of characters, and it generalizes zero-shot across novel scenes, novel characters, and novel actions. Each of those answers one of the limits. Precise control replaces predefined motion patterns, zero-shot generalization answers the unseen environments and characters, and real time answers the lag in interactivity. The aim is a world that can be manipulated as it is played, not one that can only be watched.

The groundwork is a diverse dataset spanning seven hundred and fifty scenes and one hundred and seventy-eight characters. It incorporates skeleton and geometry information as structured representations. Those representations then act as intermediate proxies that condition a video generation model. An adaptive masking mechanism is added alongside, to improve alignment between the conditioning signals and the video rollouts that are generated.

Above the generator sits an agentic framework that explicitly maintains and updates game states. That framework is what allows players to freely configure their virtual worlds and exercise fine-grained control over character motions, while visual quality and gameplay consistency are preserved.

The abstract gives no figures to weigh, and names no benchmark or baseline. Its claim is qualitative, and it is about the combination: agentic control joined to structured, proxy-based generation, which the authors present as delivering world simulation that is real-time, generalizable, and mechanically consistent.

Structure appears on both sides of the video model. Skeleton and geometry proxies shape what it generates, and an agent keeps the game state that the generated world has to stay consistent with. Player-driven game creation is the use the authors have in view, with the player setting up the world and directing who moves in it.