SWE-Game: Can Coding Agents Build the Games We Want?

SWE-Game. Can Coding Agents Build the Games We Want?. By Xiaoyu Chen and ten others, from Shanghai Jiao Tong University, Zhongguancun Academy, and four other institutions. Published on the seventh of October. No code repository is linked to it.

The title asks whether coding agents can build the games we want. Answering that takes more than reading code. A game is judged by whether its mechanics work, whether it can actually be played, and how it looks, and game development covers more than building from nothing: completing a skeleton, repairing faults and porting between engines are all part of it.

SWE-Game has two hundred and forty-seven tasks, grounded in forty-one executable reference games built in Godot. They span thirteen gameplay categories, in both 2D and 3D. Five task types cover the activities: development from a brief, implementation from a game design document, skeleton completion, repair of eighty-three injected-fault cases, and porting from Godot to Unity.

Reference materials specify the intended gameplay. A shared instrumentation interface lets drivers and probes owned by the evaluator execute actions in, and observe, games that were implemented independently. Evaluation then combines three kinds of evidence: engine-state checks, certified replay of reference inputs, and feature demonstrations authored by the agent itself. Together they assess whether mechanics are correct, whether playability is demonstrated, and, after a repair, whether behaviour was restored and preserved. Presentation is assessed separately, by vision-language rubrics written for each game.

Across six models, Opus5 achieves the highest overall score in all five task types. Even so, the best overall scores stay below sixty out of a hundred across the three construction tasks, with Brief-to-Game reaching fifty point three eight. Reviewing the submissions, the predominant implementation problems are requirement omissions and gameplay logic errors.

The evaluators were checked as well. On human-labelled behaviours from one hundred agent-built games, executable checks reach ninety-two point five nine percent balanced accuracy, against seventy-eight point four one percent for a video-based VLM judge. Rubric-based visual scores reach a Spearman correlation of zero point eight two nine with human ratings of two hundred gameplay clips.

Two results sit side by side. The strongest model leads every task type and still builds games scoring under sixty when it starts from a brief or a document. And checks that run the game agreed with human labels more closely than a judge watching video did. The authors take the whole as support for combining runtime evidence with visual assessment when the thing being judged is a game.