Phoneme-level processing in low-frequency cortical responses to speech explained by acoustic features

Abstract

When we listen to speech, we have to make sense of a waveform of sound pressure. Hierarchical models of speech perception assume that before giving rise to its final semantic meaning, the signal is transformed into unknown intermediate neuronal representations. Classically, studies of such intermediate representations are guided by linguistically defined concepts such as phonemes. Here we argue that in order to arrive at an unbiased understanding of the mechanisms of speech comprehension, the focus should instead lie on representations obtained directly from the stimulus. We illustrate our view with a strongly data-driven analysis of a dataset of 24 young, healthy humans who listened to a narrative of one hour duration while their magnetoencephalogram (MEG) was recorded. We find that two recent results, a performance gain of an encoding model based on acoustic and annotated linguistic features over a model based on acoustic features alone as well as the decoding of subgroups of phonemes from phoneme-locked responses, can be explained with an encoding model entirely based on acoustic features. These acoustic features capitalise on acoustic edges and outperform Gabor-filtered spectrograms, features with the potential to describe the spectrotemporal characteristics of individual phonemes. We conclude that models of brain responses based on linguistic features can serve as excellent benchmarks. However, we put forward that linguistic concepts are better used when interpreting models, not when building them. In doing so, we find that the results of our analyses favour syllables over phonemes as candidate intermediate speech representations visible with fast non-invasive neuroimaging.