Schwartz, J.-L. | Search bioRxiv

bioRxiv2022

Unraveling the functional attributes of the language connectome: crucial subnetworks, flexibility and variability.

Language processing is a highly integrative function, intertwining linguistic operations (processing the language code intentionally used for communication) and extra-linguistic processes (e.g., attention monitoring, predictive inference, long-term memory). This synergetic cognitive architecture requires a distributed and specialized neural substrate. Brain systems have mostly been examined at rest. However, task-related functional connectivity provides additional and valuable information about how information is processed when various cognitive states are involved. We gathered thirteen language fMRI tasks in a unique database of one hundred and fifty neurotypical adults (InLang database). The tasks were designed to assess a wide range of linguistic processes and subprocesses. From this database, we applied network theory as a computational tool to model the task-related functional connectome of language (LANG). The organization of this data-driven neurocognitive atlas of language is examined at multiple levels, uncovering its major components (or crucial subnetworks) and its anatomical and functional correlates. Furthermore, we estimate its reconfiguration as a function of linguistic demand (flexibility), or several factors such as age or gender (variability). By accounting for the multifaceted nature of language and modulating factors, this study can contribute to enrich and refine existing neurocognitive models of language. The LANG atlas can also be considered as a reference for comparative or clinical studies, involving a variety of patients and conditions.

neuroscience↗

bioRxiv2018

How predictive can be predictions in the neurocognitive processing of auditory and audiovisual speech? A deep learning study.

Sensory processing is increasingly conceived in a predictive framework in which neurons would constantly process the error signal resulting from the comparison of expected and observed stimuli. Surprisingly, few data exist on the amount of predictions that can be computed in real sensory scenes. Here, we focus on the sensory processing of auditory and audiovisual speech. We propose a set of computational models based on artificial neural networks (mixing deep feed-forward and convolutional networks) which are trained to predict future audio observations from 25 ms to 250 ms past audio or audiovisual observations (i.e. including lip movements). Experiments are conducted on the multispeaker NTCD-TIMIT audiovisual speech database. Predictions are efficient in a short temporal range (25-50 ms), predicting 40 to 60 % of the variance of the incoming stimulus, which could result in potentially saving up to 2/3 of the processing power. Then they quickly decrease to vanish after 100 ms. Adding information on the lips slightly improves predictions, with a 5 to 10 % increase in explained variance.\n\nInterestingly the visual gain vanishes more slowly, and the gain is maximum for a delay of 75 ms between image and predicted sound.

bioinformatics↗