Answering in the User's Variety: Calibrated Accommodation Pairs for Post-Training

Answering in the User's Variety. Calibrated Accommodation Pairs for Post-Training. By Zhizhuo Kou and six others, from the Hong Kong University of Science and Technology, the Hong Kong University of Science and Technology, Guangzhou, and one other institution. Published on the twentieth of October. Its code repository is worked out: about one hundred and thirty source files, with a readme and a licence.

People adjust how they speak to suit the person in front of them. Chat models tend not to. Whatever dialect a user writes in, the reply usually comes back in Standard English. That uniformity creates a measurement problem as well as a conversational one. If every user gets the same kind of answer, an audit of the outputs cannot tell whether training treated those users alike.

The authors look underneath the output. They trace how each post-training stage, from supervised fine-tuning through preference optimization with DPO to reinforcement learning, changes a model's tendency to match its user. And they read what each update prefers by comparing two replies with the same content, one matching the user and one not. Across ten open model lineages, post-training standardizes replies to every informal cue. But after adjusting for how likely each reply already was, the DPO update still favours matching British and Australian spelling over matching AAE and Singlish, on all ten lineages. Models still produce AAE features when asked to.

Calibrated accommodation pairs, or CAP, are preference data built to correct that selectivity. They reward matching the density of a user's dialect, while preserving Standard English for users who write in Standard English. The calibration is in both directions: neither ignoring the user's variety nor pushing dialect onto someone who did not use it.

Because CAP is preference data, it enters at the same stage where the selective update was found, the preference step of post-training. Each pair rewards a reply whose dialect density tracks the user's, and for a Standard-English user the rewarded reply stays in Standard English. The fix is aimed at the place the diagnosis pointed to.

On a fully open model, CAP raises convergence on dialect features it was not trained on. It does so without a detectable increase in dialect for Standard-English users, and without a cost in instruction following. The comparison point is one-sided pairs. They reach a similar gain, but only by overshooting the user and imposing dialect.

Two lessons run through the work. One is about training: preference data can teach accommodation without overshooting if it is calibrated to the user in both directions. The other is about auditing. A reply that looks the same for everyone can conceal an update that favours some varieties over others, so an audit has to read the update as well as the output.