MEA: A Reward-Driven Multi-Agent System for Faithful Model Explanations

MEA. A Reward-Driven Multi-Agent System for Faithful Model Explanations. By Yuyang Cheng and five others, from the University of Virginia, the Vellore Institute of Technology, and one other institution. Published on the first of October. No code repository is linked to it.

Machine learning models are now used across many high-stakes domains, yet they remain largely opaque to the practitioners who act on their predictions. Post-hoc explanation methods offer a view into that behaviour, but using them well takes expertise most domain experts lack: navigating high-dimensional outputs, choosing the best explanations, and synthesising evidence across separate tools. The knowledge needed to explain a model has become a barrier of its own. The people who act on a prediction are, for the most part, not the people trained to interrogate it with these tools.

MEA is a multi-agent framework meant to remove that barrier entirely. Two agents divide the work. A Proposer agent selects and configures explanation tools according to the question and the modality. An Actor agent turns the tools' outputs into natural-language explanations grounded in model behaviour, across tabular, text and vision data, and it is this agent that is optimised end to end against faithfulness. The split mirrors the three demands named a moment ago. Choosing and configuring the right tool falls to the Proposer. Reading high-dimensional output and synthesising it into an answer falls to the Actor.

Alongside the framework, the authors introduce diverse question types: feature attribution, counterfactual reasoning, and the detection of spurious features. Each is paired with a faithfulness metric based on perturbation. Optimisation then runs against faithfulness rewards, augmented with a penalty that adapts to the modality. Faithfulness, in this setting, is measured by perturbation, which ties the reward to what the model actually does rather than to how convincing the explanation sounds.

Along the way, the authors find that frontier language models systematically produce unfaithful explanations. Against that backdrop, MEA consistently outperforms post-hoc explainers, agentic baselines and closed-source baselines across six datasets.

Relative to the untrained backbone, the reward-driven optimisation improves faithfulness by twenty-eight percent on tabular data, by twenty-one percent on text, and by thirty-four percent on vision. Those three comparisons isolate the training itself. The backbone is the same model before and after, so the difference is what optimising against faithfulness adds. Taken with the earlier observation about frontier models, the pattern is consistent: untrained models drift toward unfaithful explanations, and faithful ones have to be trained for.

The broader suggestion is about where explainability lives. Instead of a fixed toolkit that the user must master, an agent sits between the question and the tools, picks among them, answers in plain language, and is trained to stay faithful to the model it explains. The authors see this as a path toward natural-language explainability that generalises beyond the fixed, single-purpose tools that have long defined the field.