Robust Explanatory Analysis for Structured Optimization Networks (REASON) in Learning

Robust Explanatory Analysis for Structured Optimization Networks (REASON) in Learning. By Vahid Jebraeeli and two others, from North Carolina State University and Nagoya University. Published on the fourteenth of October. No code repository is linked to it.

A learned model leaves some residual error, and the authors ask what that residual can tell us. Their aim is a principled way to use it: resolve the error in a hierarchy, under a composite learning objective, and in doing so identify and exploit additional information. The concern is sharpest under distribution shift and perturbation, when the evidence in hand is least likely to be enough.

The inspiration is the familiar split between System 1 and System 2 thinking, one fast and the other more deliberate. They turn it into a mechanism with two loops, both serving one shared inference goal.

A fast loop, in the System 1 role, resolves whatever part of the residual the available evidence can address. Whatever remains is the epistemic residual, and it triggers a medium speed loop in the System 2 role. That loop actively acquires additional evidence, and measures how much each piece is worth by how far it reduces the residual. The slower loop, then, only runs when the fast one leaves something unexplained. Over nested solution spaces, the authors prove the medium loop's optimum is never worse than the fast loop's.

In experiments, the combined two-loop model consistently improves on the fast loop alone, while using roughly half the computation of the medium loop. That holds under unseen corruptions too. Given a cleaner observation, it also outperforms staged classifiers and learning-to-defer methods at equal computational cost.

Leftover error becomes a signal for when to look harder. The fast path handles what it can, the slower path is called in only for what remains, and the guarantee says that calling it in can never make the answer worse.