Incidental information contaminates patient notes and disrupts clinical reasoning in large language models

Incidental information contaminates patient notes and disrupts clinical reasoning in large language models. By Krithik Vishwanath and six others. Published on the sixth of October.

Large language models are increasingly used for two clinical jobs: ambient documentation, where they write up the note from a visit, and clinical reasoning. The authors examine a failure mode the two share, which is sensitivity to information incidental to the patient encounter. Their claim is that such information gets into the notes, and that the same parts of a model that are disrupted by it also support its clinical reasoning.

In five hundred and seventy-six patient-clinician dialogues, frontier models inserted small-talk exchanges into thirty-five percent of notes. Mean quality scores barely registered this, changing by at most zero point two points on five-point scales. In three point seven percent of frontier notes, the models misattributed the asides or used them clinically. The contrast is the point: a note can carry material that does not belong in it while its quality score hardly moves, so the usual measure of a good note does not catch this.

A second test used fifty-seven mock recorded consultations. Background speech from a separate patient encounter, played at minus ten decibels, leaked into forty-eight point two percent of transcripts. Contamination was then detected in five point three percent of the downstream notes written by four open-weight models.

The mechanistic part is offered as a hypothesis. The authors propose a dual encoding account of clinical reasoning and distraction in these models, and describe the supporting evidence as preliminary: components associated with disruption by incidental information also appear to support clinical reasoning. If that holds, distraction and reasoning are not cleanly separable inside the model. The findings also come from specific setups: frontier models writing notes from dialogues, and four open-weight models working from contaminated transcripts.

The practical upshot is a test to run before deployment. Resistance to incidental information should be evaluated before clinical use, and safeguards should block contamination without stripping out the reasoning those same components support.