Duplex-MPE: Benchmarking Multi-Party Interaction in Full-Duplex Dialogue

Duplex-MPE. Benchmarking Multi-Party Interaction in Full-Duplex Dialogue. By Chengqian Ma and nine others, from Peking University, Renmin University of China, and four other institutions. Published on the twenty-fifth of September. No code repository is linked to it.

Full-duplex speech models can listen while they speak, so conversation with them does not need rigid turn boundaries. Existing benchmarks already test turn-taking, handling interruptions and multi-round dialogue. But they mostly centre on one designated user talking to the model. A different situation is an assistant taking part in a shared conversation among several people, and there the hard question is not what to say but when to say anything at all.

Duplex-MPE evaluates exactly that: when such an assistant should answer, when it should stay silent, and when it should stop speaking. Four scores cover it. One measures whether the model initiates a fresh response. One measures whether the answer is accurate. One measures whether it preserves silence when it should. And one measures whether it stops when a human resolves the request first.

The benchmark has two thousand scenarios, each with three or four human speakers and one assistant. Every scenario comes in a pair: the same request addressed to the assistant explicitly, and addressed implicitly. Models hear continuous conversation audio, with no transcripts and no turn boundaries handed to them.

Five open-weight speech systems are evaluated: MiniCPM-o 4.5, Moshi, FLM-Audio, Voila and Freeze-Omni. MiniCPM-o 4.5 leads on three of the scored capabilities. Among the others, frequent speech can go together with inaccurate answers or with failing to stay silent, so talking often is not the same as talking well.

The explicit and implicit pairing reveals a contrast. A reference built on Gemini 3.1 Pro, working from transcripts, responds sixty-four point three percentage points more often to explicit requests than to implicit ones. For the speech systems, paired tests find no significant difference in response rate between the two.

That last result is the one to carry away. A text model reading the transcript treats being named very differently from being hinted at. The speech systems, on this measure, show no significant difference between the two, which is a question the pairing was built to ask.