ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty

ProbabilityBench. Model Evaluation and Calibration with Exact Ground-Truth Uncertainty. By Yuyang Dai and three others, from Florida State University, the University of Tokyo, and two other institutions. Published on the twelfth of October. Its code repository is public and real: about twenty source files.

Evaluating a model's uncertainty on real-world data runs into a basic limit. Each instance provides a single observed label, so the true conditional probability of each class is never known. A model can report a probability, but there is no exact figure to hold it against: the label says what happened once, not how likely it was. Any judgment of whether a model's uncertainty is right has to be made against that thin evidence.

ProbabilityBench supplies the missing reference. It provides exact, instance-level class probabilities, so the quality of a model's reported probabilities can be measured directly, through KL divergence. Bayes error is computed analytically as well.

The benchmark spans three simulated text-based domains: peer review, covert information transfer, and credit risk. For each environment, the full data-generating process is defined explicitly, which is what lets both the exact class probabilities and the Bayes error be computed. Cases are then rendered into natural-language memos, and the authors give a theoretical guarantee that this rendering strictly preserves the underlying generative probabilities.

Sixteen models are evaluated. Most of the models sitting on the observed cost-quality Pareto frontiers are decision models, including Jev and Perplexity Decider. The benchmark also works one level up, as a meta-evaluation framework. Because the ground-truth values are known, it can measure the accuracy of standard calibration metrics and of Bayes-error estimators themselves.

Exactness is what everything rests on. With the true probabilities in hand, both the models and the instruments used to grade them can be checked against the answer, rather than against a single label that only hints at it.