DyCraft: An Agentic System for Long-Horizon 4D Scene Generation

DyCraft. An Agentic System for Long-Horizon 4D Scene Generation. By Jeff Tan and seven others, from Stanford University and the University of Cambridge. Published on the twenty-ninth of October.

Frontier coding agents can already generate high-quality 3D scenes and articulated assets. Where they stumble is time. Asked for a coherent animation that runs over a long horizon, an agent working with a general-purpose harness frequently produces something physically implausible: objects that pass through one another, objects with nothing holding them up, motion that looks wrong. The abstract locates the reason in how such a bare agent works. It defaults to scripting the whole animation in a single pass. Written that way, the result is hard to verify, and the tangle of spatial and temporal constraints is hard to pull apart, so once an error is in, the agent struggles to repair it.

DyCraft borrows from the practice of human animators. Rather than one pass over everything, it breaks animation into steps that a coding agent can solve more reliably. A large, entangled problem becomes a sequence of smaller ones, each of which can be checked before the next begins.

The pipeline opens by expanding the user's prompt into a detailed plan and building a static scene. Next comes a 3D storyboard of key animation poses. Each frame of that storyboard is itself a static 3D scene, which means the agent can build it and debug it in isolation, with no motion to worry about yet. Storyboarding turns the animation into pairs of start and end states, both verified. Between each pair, the agent authors local motion using a collection of motion authoring tools. The hard global question, does this whole sequence hold together, is replaced by many local ones: get from this checked pose to that checked pose.

Evaluation covers twenty-four long-horizon, multi-shot animation tasks. They span humanoid robots, human and animal characters, and physical mechanisms. The comparison is against a native GPT-6 agent running in Codex, along with prior agentic baselines. Against those, DyCraft substantially reduces physical violations. Its animated 3D scenes are of higher quality, and human raters prefer them more often.

Nothing here asks the agent to be smarter in one shot. The gain comes from restructuring the work so that every intermediate state is a static scene the agent already knows how to build and check. Motion is only ever authored between two poses that have both passed verification, and that ordering, taken from how animators work, is what lets errors be caught where they start.