Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text

Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text. By Dulhan Jayalath and one other, from the University of Oxford. Published on the thirtieth of September. Its code repository is public and real: about thirty source files.

Decoding words from brain recordings taken outside the skull has seen major reported improvements. This paper finds that much of that improvement can be reproduced without any brain data at all. The networks had found a shortcut: the timing of the words. And once that shortcut is removed, the authors argue, decoding does not just get more honest. It gets better.

The case centres on the influential work of d'Ascoli and colleagues. There, brain activity recorded while subjects listen to continuous speech is cut into fixed-length windows, one starting at each word, and a neural network predicts all the words of a sentence together. Neighbouring windows partly overlap, and that overlap quietly reveals the interval between one word and the next.

That interval tells you how long each spoken word lasted, and different words have different durations. The word "the" is far shorter than "supercalifragilisticexpialidocious". So the network can improve its guesses without leaning on the brain activity underneath. The numbers bear this out. On synthetic signals that contain no brain information, the method reaches twenty-two percent balanced accuracy. On real brain recordings it reaches twenty-two point three percent.

The fix is one simple change. Instead of encoding all the windows in a sentence jointly, each is processed on its own. Denied the timing cue, the network does better, because it learns word-specific information from the brain recordings themselves. That in turn makes two existing strategies far more effective than before: aggregating predictions from distinct neural responses to the same word, and using a pretrained LLM as a linguistic prior. Both now substantially improve results. The combined recipe, SimpleB2T, reaches a word error rate of thirty-six point six percent with five observations per word on the authors' perceived speech benchmark.

That word error rate is measured on perceived speech, the subject listening rather than speaking, and with five observations of each word, not one. The authors describe it as approaching past invasive speech decoding performance, and they add the qualifier themselves: under different conditions. The shortcut argument is made against one influential pipeline, with fixed windows and sentence-level joint prediction, and the claim is that the reported gains there are largely reproducible without brain data, rather than that every non-invasive decoder is affected.

Twenty-two percent without a brain, twenty-two point three with one. That gap is the whole warning, and following it up pays twice. It exposes the shortcut in brain-to-text decoding, and closing it turns out to be the simpler and considerably more effective path, because the model finally has to listen to the brain.