DialFork: Going Beyond Factuality to Evaluate Procedural Hallucination in Conversational Agents

DialFork. Going Beyond Factuality to Evaluate Procedural Hallucination in Conversational Agents. By Sergio Burdisso and six others. Published on the fifteenth of October. Its code repository is worked out: about fifty-five source files, with a readme and a licence.

Language models deployed as conversational agents are increasingly expected to follow procedural logic: to move through a conversation in the order a process requires. Factual hallucination has received extensive attention. A different failure has been overlooked. The authors call it procedural hallucination. An agent produces utterances that are each fluent and factually plausible, while the conversation as a whole drifts away from the intended process. Checking facts one utterance at a time would not see it.

DialFork is presented as the first benchmark to measure this failure on raw dialogue text. It defines three evaluation tasks, each at a different granularity. The first asks how far a set of dialogues deviates from conformant ones, which serves monitoring of deployed agents. The second asks how far a single dialogue deviates, framed as procedural retrieval. The third asks at which turn a dialogue first goes off track, the setting of real-time guardrails. The three move from the population, to the single conversation, to the single turn. Together they cover watching a deployed agent in aggregate, finding the dialogues that strayed, and catching the moment a conversation leaves its process while it is still happening.

The construction starts from conformant dialogues. Each is forked at a controlled turn position, and one of four hallucination types is injected from there. The result is thirty-six thousand two hundred and seventy-eight dialogues, covering twenty-four tasks in twelve domains. Thirty human annotators validate the construction. Because each fork starts from a conformant dialogue at a known turn, every hallucinated dialogue comes with its conformant counterpart and the position where the two part ways.

For the first two tasks the authors also propose metrics that are aware of dialogue flow, called FlowKL and FlowScore. They compare dialogues by how the dialogues unfold, rather than by what they contain. That choice is the benchmark's premise turned into a measurement: if the failure lives in the process, the comparison has to look at the process.

At the dialogue level, FlowScore significantly outperforms all baselines, and it does so across seven encoders. The authors draw a direct conclusion from that: semantic similarity is no substitute for procedural similarity. Two conversations can sound alike and still follow different processes.

DialFork is released under the MIT license. Its argument is that conversations have a shape as well as a content, and that an agent can get every fact right while still taking the wrong path. The authors hope the community will measure conversational process alongside content.