TurnTurk-GPT: Text-Based Turn-Taking Prediction for Turkish

TurnTurk-GPT. Text-Based Turn-Taking Prediction for Turkish. By Ahmet Tuğrul Bayrak and three others, from Ata Technology Platforms. Published on the thirtieth of October.

A conversational system has to judge when the person speaking has finished a turn. This work takes up that judgment for Turkish, from text alone, and under realistic conditions: text that is unpunctuated and lower-cased, close to what a streaming speech recogniser would produce. No punctuation marks the end of a sentence, and no capital letter marks the start of the next, so whatever cue the model uses has to come from the words themselves.

TurnTurk-GPT is a text-based turn-taking predictor for Turkish. It is obtained by fine-tuning a Turkish GPT-2 model with the TurnGPT objective, on a mix of synthetic and real dialogue.

The evaluation runs on real Turkish conversation, against three kinds of competitor: rule baselines, a zero-shot Turkish language model, and a widely used multilingual end-of-turn detector. Decision points are placed at the level of each word. Folds are split two ways, disjoint by recording and disjoint by speaker. Thresholds are chosen by nested selection, and differences are checked with paired cluster bootstrap tests. The two kinds of split answer different questions: whether the model generalizes to new recordings, and whether it generalizes to people it has never heard.

TurnTurk-GPT outperforms the multilingual detector under both kinds of split. Against the zero-shot language model the picture is narrower. It wins only when the test speakers also occur in training, and on unseen speakers the two are statistically indistinguishable.

An ablation shows where the signal lives. Removing the suffixes of the turn-final word reduces AP by up to forty-four percent, against two percent for a word inside the sentence. The Turkish end-of-turn cue, on this evidence, sits in verb-final morphology. Synthetic dialogue alone gives the weakest trained model, and when a speaker pauses, the zero-shot and multilingual baselines stay close to chance.

The model and the evaluation protocol are both publicly released. Two results travel together: a predictor that beats a widely used multilingual detector, and a finding about the language itself, that a finished turn in Turkish is marked in the suffixes of its final word.