Hadjinicolaou, A. E. | Search bioRxiv

bioRxiv2025

Decoding semantics from natural speech using human intracranial EEG

Brain-computer interfaces (BCIs) hold promise for restoring natural language production capabilities in patients with speech impairments, potentially enabling smooth conversation that conveys meaningful information via synthesized words. While considerable progress has been made in decoding phonetic features of speech, our ability to extract lexical semantic information (i.e. the meaning of individual words) from neural activity remains largely unexplored. Moreover, most existing BCI research has relied on controlled experimental paradigms rather than natural conversation, limiting our understanding of semantic decoding in ecological contexts. Here, we investigated the feasibility of decoding lexical semantic information from stereo-electroencephalography (sEEG) recordings in 14 participants during spontaneous conversation. Using multivariate pattern analysis, we were able to decode word level semantic features during language production with an average accuracy of 21% across all participants compared to a chance level of 10%. This semantic decoding remained robust across different semantic representations while maintaining specificity to semantic features. Further, we identified a distributed left-lateralized network spanning precentral gyrus, pars triangularis, and middle temporal cortex, with low-frequency oscillations showing stronger contributions. Together, our results establish the feasibility of extracting word meanings from neural activity during natural speech production and demonstrate the potential for decoding semantic content from unconstrained speech.

neuroscience↗

bioRxiv2023

Natural language processing models reveal neural dynamics of human conversation

Through conversation, humans relay complex information through the alternation of speech production and comprehension. The neural mechanisms that underlie these complementary processes or through which information is precisely conveyed by language, however, remain poorly understood. Here, we used pretrained deep learning natural language processing models in combination with intracranial neuronal recordings to discover neural signals that reliably reflect speech production, comprehension, and their transitions during natural conversation between individuals. Our findings indicate that neural activities that encoded linguistic information were broadly distributed throughout frontotemporal areas across multiple frequency bands. We also find that these activities were specific to the words and sentences being conveyed and that they were dependent on the words specific context and order. Finally, we demonstrate that these neural patterns partially overlapped during language production and comprehension and that listener-speaker transitions were associated with specific, time-aligned changes in neural activity. Collectively, our findings reveal a dynamical organization of neural activities that subserve language production and comprehension during natural conversation and harness the use of deep learning models in understanding the neural mechanisms underlying human language.

neuroscience↗