Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models

Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models. By Zikai Zhou and four others, from Stanford University, Duke University, and two other institutions. Published on the twenty-sixth of October. Its code repository is worked out: about sixty-five source files, with a readme and a licence.

For a language model to be explainable and accountable, it has to reveal how external information shaped its answer. The paper frames this as a question of self-attribution. A model is self-attributive if it indicates when its response to a question was influenced by information supplied alongside that question, and only when. Both halves count. Staying silent about an influence is a failure, and so is crediting information that made no difference.

Measured against that standard, most of the eleven frontier models examined rarely attribute the information that actually influenced them, and the rest attribute information that did not. One natural fix does not work. Training acknowledgment into the model's reasoning leaves the problem in place, because the acknowledgment does not reach the final answer.

The proposed route is counterfactual training. It explicitly teaches a model to tell apart answers that depend on supplied information from answers that do not. The method, called CREDIT, applies this directly where the earlier attempt fell short: it supervises self-attribution in the final answer itself, not in the reasoning that precedes it.

The supervision comes from counterfactual labels, so no human annotation is needed. The labels record whether an answer depends on the supplied information, which is exactly the distinction the model is being asked to report.

CREDIT substantially improves attribution in four open models, two of them distilled reasoning models. It outperforms both reasoning-level fine-tuning and prompting baselines. One case shows the distinction at its sharpest. Among answers that agree with a user's suggestion, CREDIT separates the ones the suggestion actually changed from the ones the model would have given anyway. On the surface those two kinds of answer look identical. The gains also transfer, partly, to information types not seen in training, and to prompts containing sources that support conflicting answers.

The thread through the paper is about where the signal has to live. Asking a model to acknowledge influence in its reasoning did not carry through to what it finally said. Supervising the final answer, with labels built from what would have happened without the information, is what moved attribution, and it did so without a human labelling anything.