Search bioRxiv⌕ Search

Biology subjects

Ottenhoff, M.

Publications and source records attributed to Ottenhoff, M..

2 recordsLinked to original sources

Whole-brain dynamics of articulatory, acoustic and semantic speech representations

Speech production is a complex process that traverses several representations, from the meaning of spoken words (semantic), through the movement of articulatory muscles (articulatory) and, finally, to the produced audio waveform (acoustic). In our study, we aimed to identify how these different representations of speech are spatially and temporally distributed throughout the depth of the brain. By considering multiple representations from the same exact data, we can limit potential con-founders to better understand the different aspects of speech production and acquire crucial complementary information for speech brain-computer interfaces (BCIs). Intracranial speech production data was collected of 15 participants, recorded from 1647 electrode contacts, while they overtly spoke 100 unique words. The electrodes were distributed across the entire brain, including sulci and subcortical areas. We found a bilateral spatial distribution for all three representations, although there was a stronger tuning in the left hemisphere with a more widespread and temporally dynamic distribution than in the right hemisphere. The articulatory and acoustic representations share a similar spatial distribution surrounding the Sylvian fissure, while the semantic representation appears to be widely distributed across the brain in a mostly distinct network. These results highlight the distributed nature of the speech production process and the potential of non-motor representations for speech BCIs.

neuroscience↗

Real-time Synthesis of Imagined Speech Processes from Minimally Invasive Recordings of Neural Activity

Speech neuroprosthetics aim to provide a natural communication channel to individuals who are unable to speak due to physical or neurological impairments. Real-time synthesis of acoustic speech directly from measured neural activity could enable natural conversations and significantly improve quality of life, particularly for individuals who have severely limited means of communication. Recent advances in decoding approaches have led to high quality reconstructions of acoustic speech from invasively measured neural activity. However, most prior research utilizes data collected during open-loop experiments of articulated speech, which neglects the critical human-in-the-loop aspect of a practical speech neuroprosthetic. Here we present an approach that synthesizes audible speech in real-time for both imagined and whispered speech conditions. Using a participant implanted with stereotactic depth electrodes, we were able to reliably generate audible speech in real-time. The decoding models rely predominately on frontal activity suggesting that speech processes have similar representations when vocalized, whispered, or imagined. Our real-time synthesis approach represents an essential step towards investigating how patients will learn to operate a closed-loop speech neuroprosthesis, as well as the development of techniques that incorporate co-adaptation of the user and system for optimized performance.

neuroscience↗