EPSBench: Benchmarking LLM Agents for Earnings Forecasting

EPSBench. Benchmarking LLM Agents for Earnings Forecasting. By Yanbin Yin and four others, from the Georgia Institute of Technology. Published on the twenty-eighth of October.

Forecasting a company's quarterly earnings is not a single guess at a final number. An agent has to connect financial evidence to the income-statement line items that actually produce the earnings the company reports. That connecting ability is what EPSBench is built to study.

The benchmark covers one hundred and forty-nine firm-quarters, drawn from eighty-six US companies. For each one, it measures two things. First, point forecast error, judged against the date-matched analyst consensus. Second, calibration: whether the agent's prediction intervals actually achieve the coverage they claim.

Agents work from a fixed collection of filings, earnings-call transcripts and news. Rather than outputting earnings directly, they estimate the company-specific income-statement drivers, each with cited evidence. A shared accounting calculator then converts those estimates into GAAP diluted earnings per share. Because every agent goes through the same calculator, the comparison is between forecast assumptions, not between ways of doing the arithmetic. Six models are evaluated under a common harness.

On point forecasts, three of the six models show no detectable paired difference in error from the analyst consensus. The other three have higher error.

The intervals fall short of their stated coverage. Across all six models, intervals labelled as eighty percent intervals contain the reported earnings in only thirty-seven to seventy percent of the runs that return an interval. Most of the misses fall above the upper bound, because the forecasts run low.

The driver records also explain the spread between models: most of the error gap traces to their operating-income estimates.

Forcing the forecast through line items is what makes the result diagnosable. A bare earnings number can only be right or wrong. A set of cited drivers fed through a common calculator shows where the forecast went wrong, and here it points at operating income, and at intervals that are both too confident and pitched too low.