Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds

Scaling in Games. Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds. By Kuan Zhang and ten others, from miHoYo, Tsinghua University, and four other institutions. Published on the twentieth of October. Its code repository is public but slight: eight source files.

In physical environments, scaling data, diversity and model capacity has improved embodied learning, and there the varied experience is all grounded in the same physical laws. Virtual worlds break that guarantee. They differ in visual style, dynamics, controls and objectives. The claim this study makes is that in such worlds the three scaling axes do different jobs. More data mainly helps in the worlds already in the corpus. More diversity helps transfer to related worlds, but gains little in distant ones. And more model capacity buys data efficiency, while generalising worse to distant worlds when training comes from a single world.

The evidence is a controlled study of continued pre-training along those three axes: data volume, diversity, and model size. There are one hundred and twenty training configurations. They span four model sizes, from zero point eight billion to twenty-seven billion parameters.

They also span six data budgets, from fifty hours to one thousand six hundred hours, and five mixtures, ranging from Minecraft-only data to diverse multi-world corpora. Performance and generalisation are assessed with validation losses at three levels and with gameplay benchmarks, reaching from worlds inside the corpus out to unseen ones.

Training shows two phases. Early on there is an interface phase with broad benefits across worlds. Later comes a world phase, in which gains narrow to the in-corpus and related worlds. The authors fit an empirical scaling law relating validation loss to data volume, diversity and model size. In use, the resulting models are competitive with Minecraft specialists and with frontier models in 3D worlds, and they perform best in real-time settings.

The limits are stated plainly. Gains in distant worlds remain small, and the penalty for larger models under single-world training shows up exactly there. The study is about continued pre-training, which is a form of imitation, and the authors point to the limits of imitation and to the persistent gaps in distant worlds as evidence that continued pre-training is not the whole answer. Building generalist embodied agents, in their reading, requires richer training stages beyond it.

What carries over from physical-world scaling is real but partial. Volume, diversity and size each pay, but each pays in a different place, and the far edge of the distribution, the worlds least like the training mix, is where none of them pays much. The scaling law and the two-phase picture give a map of where the returns are. The distant worlds mark where the map runs out.