AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents

AlphaBet. Reinforcement Learning for LLM Event Forecasting Agents. By Liqin Ye and seven others, from the Georgia Institute of Technology and Amazon. Published on the fifteenth of October.

Forecasting an event that has not yet resolved takes two skills. You have to find the relevant evidence, and you have to decide how strongly that evidence supports each outcome. Training Large Language Model forecasters on outcomes often skips the first skill. The model is handed evidence that has already been assembled, at a single prediction date, which leaves evidence acquisition outside the learning loop entirely.

AlphaBet brings it back inside. It is an agentic reinforcement learning framework that trains a single LLM policy to search, to reason, and to report probabilities. Crucially, it does this across multiple dates before the event resolves. Each dated forecast is scored against the eventual outcome with a proper scoring rule. That means one resolved event can supervise decisions made under several different states of information.

The authors also take forecasting regret apart. They decompose it into two pieces: an evidence-acquisition gap and a probability-reporting gap. That split lets them examine a detail of the training algorithm. They show that Group Relative Policy Optimization, known as GRPO, has a standard-deviation normalization that can distort the reporting target when the scores are multi-class proper scores.

The base model is GLM-4.5-Air. On Forecast-Dojo, AlphaBet reduces the Brier score from zero point seven three one to zero point six two six. On FutureX it falls from zero point six one two to zero point five two eight. Lower is better for this score. Controlled ablations attribute the gains to multi-date training, and show they come from improvements in both information acquisition and probability reporting. The experiments also back the claim about GRPO: its standard-deviation normalization degrades performance on proper scoring rules, while leaving accuracy on the top outcome largely unchanged.

A forecaster is judged on its probabilities, not just on which outcome it ranks first, and this work keeps returning to that distinction. Training at several dates puts evidence gathering back into the loop, and the GRPO result warns that a normalization step can quietly damage calibrated reporting even when headline accuracy looks the same.