Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players

Build, Don't Play. Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players. By Changlian Ma and one other, from Nanjing University and the Shanghai AI Laboratory. Published on the twenty-fourth of October. Its code repository is worked out: about one hundred and forty source files, with a readme and a licence.

The question is whether LLM agents can carry a hard real-world problem all the way from raw data to a system that holds its own against people. The testbed chosen to ask it is Hearthstone Battlegrounds, an eight-player auto-battler. Its shops and battles are stochastic, information is hidden, and nearly a thousand cards interact with one another. That is a long way from a tidy benchmark, and getting from nothing to a competitive player means building a simulator, opponents and a policy along the way.

The answer proposed is AutoTavern, a self-improving research system. Several specialised LLM research agents share a typed research memory. Every claim they make has to pass verification gates. Live games close the loop, so the same feedback lets the agents improve the simulator, the opponents and the policy, and also improve their own research process. A human stays involved in a narrow role: setting goals, and ruling on game mechanics that are ambiguous. Everything else, from the code of the simulator to the training of the player, is left to the agents. The self-improving part is meant literally. The loop that tunes the policy is also the loop that tunes how the agents go about their research, so the system that does the work and the way it works are both being revised by the same stream of results from real games.

Over twenty days, AutoTavern built TavernGym, described as the first faithful, full-content reproduction of the live game. On top of it the agents delivered twenty-three versions of TavernAgent. Those were trained with PPO, against opponents calibrated to real games. The agents, in other words, did not play the game themselves. They built the thing that plays it.

TavernAgent then plays complete games on the live ladder by itself. Its rating rose from six thousand to about fifteen thousand. On the public leaderboard's own measure, its last twenty records average a placement of one point six, with eighty percent first places. The server's ten highest-rated players manage an average placement of two point three four and fifty-nine percent first places, so TavernAgent is ahead on every statistic.

A second test is TavernBench: nine hundred and sixty tasks spread over forty scenarios. TavernAgent answers ninety-one percent of the play tasks. Frontier LLMs answer at most sixty-three percent. Speed separates them further. TavernAgent decides in zero point zero three seconds, while the LLMs need up to twenty-six seconds. Held to the one second a live turn allows, TavernAgent keeps ninety percent, and no LLM keeps more than twenty-six percent.

The authors compress all of this into one line: LLM agents build far better than they play. The frontier models lose to the system they constructed, both on accuracy and on time. TavernBench and TavernGym are to be open-sourced. What the paper leaves you with is a division of labour, where the language model is the researcher and the engineer, and something faster, trained by it, sits at the table.