Build, Don’t Play: How AutoTavern Turns LLM Agents into AI System Builders

Introduction

Large language models can struggle when they must directly control a complex game. They have to interpret a changing state, plan across many turns, account for randomness and hidden information, and respond within a strict time budget. A new study, Build, Don’t Play, explores a different division of labor: let LLMs act as researchers and engineers, then let a specialized reinforcement-learning system play the game.

That is the role of AutoTavern, a self-improving research system built around Hearthstone Battlegrounds, an eight-player auto-battler with stochastic shops and battles, hidden information, and nearly a thousand interacting cards.

Key points

• Specialized agents share research memory. AutoTavern assigns different research responsibilities to LLM agents. Their findings are stored in a typed research memory, while claims about game mechanics pass through verification gates. A human remains involved when the rules are ambiguous, setting goals and boundaries rather than manually implementing every detail.
• The simulator comes before the player. In 20 days, the system built TavernGym, intended as a faithful, full-content reproduction of the live game. This is important because reinforcement learning is only as reliable as the environment used for training. Random shops, combat outcomes, hidden information, and card interactions all need to be represented well enough for policy improvements to transfer to real matches.
• Live games close the loop. AutoTavern uses real-world feedback to refine not only the policy, but also the simulator, opponent models, and research process itself. It produced 23 versions of TavernAgent and trained them with PPO against opponents calibrated to real games.
• The gains are both strategic and operational. TavernAgent rose from about 6,000 to roughly 15,000 rating on the live ladder. Under the public leaderboard’s reported measure, its last 20 games averaged a placement of 1.60 with 80% first-place finishes, compared with 2.34 and 59% for the server’s ten highest-rated players in the supplied comparison. On TavernBench, which contains 960 tasks across 40 scenarios, TavernAgent achieved 91% accuracy on play tasks and made decisions in about 0.03 seconds.

Why it matters

The central result is not simply that an AI player achieved a strong game score. AutoTavern links rule discovery, environment construction, opponent generation, policy training, evaluation, and real-world correction into one research loop. LLMs handle exploration, coding, verification, and coordination—activities where broad knowledge and flexible reasoning are useful. A trained specialist handles the time-critical action selection.

This helps explain the paper’s title. Directly playing a game is not the same as building a system that plays it well. In TavernBench, frontier LLMs scored no more than 63% on the reported play tasks, and some required up to 26 seconds to decide. Within the one-second window available in a live turn, TavernAgent retained 90% performance, according to the supplied results. The gap points toward a practical architecture for future agents: a general model may design and improve a task-specific system instead of being placed directly in the control loop for every decision.

The findings should still be interpreted in context. The experiment focuses on one game, human guidance remains necessary for ambiguous mechanics, and the quality of simulator and opponent calibration affects training. AutoTavern is therefore best understood as evidence for an automated AI-development loop, not as proof that LLMs can independently solve every open-ended research problem.

Source: Hugging Face Daily Papers