Choosing LLMs for Strategic Agent Interactions

Choosing LLMs for Strategic Agent Interactions. By Arastun Mammadli and three others, from the University of Exeter and IBM Research. Published on the thirteenth of October. No code repository is linked to it.

Game-theoretic benchmarks claim that how well a large language model makes strategic decisions varies with the type of game, the structure of the interaction, and the capability being tested. But those benchmarks only score individual, fixed models, one at a time. They tell you how each model plays. They do not tell you how to choose among models when the choice itself is part of the game.

Meanwhile, a separate line of work has built LLM routers, which choose between models by trading quality against cost. Routers have no notion of strategic structure at all. They do not account for an opponent who might be unknown, and who can adapt over time and shift the very feedback signal the router is learning from. So two lines of work sit side by side, and neither covers the case where a model has to be picked against an adversary that is itself changing.

The authors extend game-theoretic evaluation to cover that missing case. They define strategic selection as a meta game, layered on top of the base game. In each round, a strategic selector fields one model from a portfolio of models. That model then has to navigate a game whose payoff matrix is latent, which means it is not given to anyone in advance and has to be learned online, round by round. What gets measured is how well the selector does over the course of play, not how well any single model does in isolation.

A first finding concerns the measurement itself. The evaluation signal turns out to be specific to each game, and extracting a reliable signal requires a much higher number of seed runs than traditional benchmarking setups use. A score drawn from only a few runs would not be one to trust.

The evaluation spans five games and ten models. On top of that, the authors implement a suite of selector methods: some that rely on query embeddings, some that rely on a model's self-reflection, and a lightweight multi-armed bandit. They also test simple non-stationary and adversarial variants of the bandit against a stationary bandit baseline, since a shifting opponent is the condition those variants are meant for.

A model's strategic competence varies significantly with the game setup and with the opponent it faces. No single model is simply the strategic one, which is exactly why selection becomes a real problem.

Among the selectors, the approaches built on query embeddings or on self-reflection do not consistently capture the underlying strategic signal. The lightweight multi-armed bandit performs more reliably across games, though it still falls short of the best fixed model. And the non-stationary and adversarial bandit variants fail to outperform the stationary baseline.

The result points to an open gap rather than a solution. Choosing which model to send into a game against an adaptive opponent is its own problem, distinct from ranking models and distinct from routing for cost. The methods tried here, including the ones designed for change, do not yet handle an opponent that moves.