Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings
Shantanu Vispute, Aditya Mishra, Siddhartha Saxena
Foyer
Published September 30, 2026 · on the feed October 1, 2026 · ▲ 2 · arXiv 2609.39344
The abstract
Speech transcripts used as long-term memory must preserve both words and stable speaker identities. Existing meeting-transcription metrics either ignore speakers or remap anonymous speakers independently in each recording, so they cannot measure whether the same person retains one identity across meetings. We evaluate persistent speaker attribution with Speaker Identified cpWER (SI-cpWER), which scores a corpus under one global speaker-ID assignment. The benchmark covers five commercial diarize-then-identify cascades, two open academic baselines, and ThyVoice on the full 129-meeting CHiME-8 NOTSOFAR evaluation set in clean and noiseaugmented form, plus CHiME-6. ThyVoice is our end-to-end reference system; it repairs overlap and gates the evidence used to create and update voiceprints. Requiring persistent identity changes the commercial ranking: ThyVoice records lower SI-cpWER than every evaluated commercial cascade in all three conditions and the lowest mean in the full panel, 47.13 versus 54.75 for the next system. Complementary lexical, diarization, per-recording attribution, and speaker-clustering diagnostics characterize upstream error surfaces in the final attributed record. These results show why persistent attribution must be evaluated directly in systems that reuse conversations across time.
The Brief
2:05A short podcast summary of the paper. Written and voiced by AI.
Transcript
Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings. By Shantanu Vispute and two others, from Foyer. Published on the thirtieth of September. No code repository is linked to it.
When speech transcripts serve as long-term memory, they have to keep two things intact: the words, and who said them, with each speaker holding a stable identity. Current metrics for meeting transcription fall short of the second. Some ignore speakers altogether. Others remap anonymous speakers separately inside each recording. Neither can tell whether the same person keeps a single identity from one meeting to the next.
The measure proposed is Speaker Identified cpWER, shortened to SI-cpWER. Its defining choice is scope. It scores a whole corpus under one global assignment of speaker identities, so a person who is labelled one way in an early meeting and another way in a later one is penalised for it. Words and identities are judged together, across the whole collection, rather than recording by recording.
The panel includes five commercial cascades that diarize first and identify afterwards, two open academic baselines, and ThyVoice, the authors' own end-to-end reference system. ThyVoice repairs overlapping speech and gates the evidence it uses to create and update voiceprints. Everything is run on the full CHiME-8 NOTSOFAR evaluation set of one hundred and twenty-nine meetings, both clean and with added noise, and on CHiME-6 as well.
Demanding persistent identity reshuffles the commercial ranking. ThyVoice has a lower SI-cpWER than every commercial cascade in all three conditions, and the lowest mean across the full panel: forty-seven point one three, against fifty-four point seven five for the system in second place. Further diagnostics, covering lexical errors, diarization, attribution within single recordings, and speaker clustering, trace where upstream errors surface in the final attributed record.
A transcript that gets every word right can still fail as memory if the speaker it credits drifts between sessions. The authors' conclusion follows from that: systems which reuse conversations across time need persistent attribution measured directly, because per-recording scores do not reveal it.
Paper Quest · 7 chapters · about 39 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A transcript that remembers needs to remember who, not just what.
You are building an assistant that remembers meetings, so that later you can ask what a colleague decided last week. The words must be right, and so must who said them.
The road ahead
- 1 · The problem ~6 min A meeting memory needs the right words and the right speaker, but today's scores never check that a person keeps one label from meeting to meeting.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min SI-cpWER scores the whole collection under one speaker mapping, so a person whose label changes between meetings now costs errors.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Run every system on all meetings, choose one global speaker assignment for the corpus, then score attributed words across the whole corpus.
- 6 · The experiment ~6 min You’ll guess: Scored with one global speaker mapping, how did ThyVoice fare against the commercial diarize-then-identify cascades across the three test conditions?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.