WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State

WorldStream. Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State. By Xiaoxiao Ma and thirteen others, from the University of Science and Technology of China and Joy Future Academy at JD. Published on the twentieth of October. Its code repository is public but slight: four source files.

An open-ended 3D world model is asked here to do three things at once. It should extend playable terrain beyond any map fixed in advance. It should act on open-ended language while the world is running. And when the user walks back to a place already visited, that place should return with its edits intact. Each of those pulls on the others: a world that keeps growing has to stay connected, has to stay within a budget, and has to remember.

WorldStream answers with two pieces. The first is a hierarchical planner over a typed decision model, which emits structured decision state rather than geometry. The second is a single world record that holds everything the world needs to be rebuilt and acted on. Plans decide what a region should be; the record keeps what it has become.

The planner works at two levels. Macro fixes settlement, ecology, transport, and hydrological structure across a group of tiles. Micro then specializes that contract into the semantics of one tile, under measured terrain and the commitments neighbouring tiles have already accepted. Deterministic local code compiles the accepted plans into terrain, roads, settlements, vegetation, and entities.

Streaming gives each stage a different reach. Plans run ahead along the corridor the user is predicted to take. Compilation happens only where movement will soon need it. Geometry behind the user is released. The world record retains plans, seeds, object identities and mutable state, boundary commitments, and edits by attribute, and it is revised as the user advances and edits. A returning tile is recompiled from that record. The same record grounds interaction: the model can act on existing entities, including stateful NPCs that update as they act, edit the present region, or attach intent to regions that have not yet been planned.

The abstract states the outcome in terms of the three capabilities together. The world expands under bounded budgets, stays connected across tiles that were planned independently, and remains interactive. It reports no figures for these properties.

The design separates deciding from building and building from remembering. Planning reaches furthest ahead, geometry exists only near the user, and the record is what persists, so that a place can be released and later rebuilt with its edits, while language acts on that same record.