Generative World Simulation with DAO and JING

Generative World Simulation with DAO and JING. By XGEN Team and eighteen others, from the XGEN Team. Published on the fourteenth of October. Its code repository is public and real: about thirty source files.

An interactive world needs two things at once. It needs a world that remembers: where characters are, what has been moved, what happened earlier and what it led to. And it needs a continuous first-person experience of that world, seen and heard by whoever is in it. The aim here is to couple the two, so that sustained interaction, by humans and by agents, plays out against a persistent world state that everyone shares.

The Generative World Simulation framework splits the job between two components. DAO is a computable world engine. JING is an Egocentric Interactive Experience Model. DAO holds the world. JING produces what each observer sees and hears in it. They are joined in a continuous interaction loop, so the state shapes the experience and actions taken inside the experience feed back into the state.

DAO maintains the states of characters, objects and spaces, along with the relationships among them. It advances the world according to interaction rules and world time, while preserving the consequences of past actions. From that evolving state, DAO derives conditions specific to each observer and hands them to JING. JING combines those conditions with references for characters, objects and the scene, with the observer's history of observations, and with the current actions, and from all of that generates the observer's audiovisual experience.

What the authors present is the interaction this makes possible. Users can explore environments, manipulate objects, and communicate with characters, steering through viewpoint control and text instructions. Their actions shape the shared state, and through it, the experiences that come after.

The design choice at the centre is the separation. World-state evolution lives in one place, experience generation in another, and the loop between them is what lets the result be, in the authors' framing, a world that retains its history, evolves through action, and persists beyond any individual observer. Leave the scene, and the world you left keeps its state for the next one to arrive.