Search bioRxiv⌕ Search

Biology subjects

King, J.-R.

Publications and source records attributed to King, J.-R..

18 recordsLinked to original sources
bioRxiv2024

Modality-Specific and Amodal Language Processing by Single Neurons

According to psycholinguistic theories, during language processing, spoken and written words are first encoded along independent phonological and orthographic dimensions, then enter into modality-independent syntactic and semantic codes. Non-invasive brain imaging has isolated several cortical regions putatively associated with those processing stages, but lacks the resolution to identify the corresponding neural codes. Here, we describe the firing responses of over 1000 neurons, and mesoscale field potentials from over 1400 microwires and 1500 iEEG contacts in 21 awake neurosurgical patients with implanted electrodes during written and spoken sentence comprehension. Using forward modeling of temporal receptive fields, we determined which sensory or abstract dimensions are encoded. We observed a double dissociation between superior temporal neurons sensitive to phonemes and phonological features and previously unreported ventral occipito-temporal neurons sensitive to letters and orthographic features. We also discovered novel neurons, primarily located in middle temporal and inferior frontal areas, which are modality-independent and show responsiveness to higher linguistic features. Overall, these findings show how language processing can be linked to neural dynamics, across multiple brain regions at various resolutions and down to the level of single neurons.

neuroscience↗
bioRxiv2024

Algebraic structures emerge from the self-supervised learning of natural sounds.

Humans can spontaneously detect complex algebraic structures. Historically, two opposing views explain this ability, at the root of language and music acquisition. Some argue for the existence of an innate and specific mechanism, like "merge" (Chomsky) or "neural recursion" (Dehaene). Others argue that this ability emerges from experience (e.g. Bates): i.e. when generic learning principles continuously process sensory inputs. These two views, however, remain difficult to test experimentally. Here, we use deep learning models to evaluate the factors that lead to the spontaneous detection of algebraic structures in the auditory modality. Specifically, we train multiple deep learning models with a variable amount of natural sounds and a self-supervised learning objective. We then expose these models to the experimental paradigms classically used to evaluate the processing of algebraic structures. Like humans, these models spontaneously detect repeated sequences, probabilistic chunks and complex algebraic structures. Also like humans, this ability diminishes with structure complexity. Importantly, this ability can emerge from experience alone: the more the models are exposed to natural sounds, the more they spontaneously detect increasingly complex structures. Finally, this ability does not emerge in models pretrained only on speech, and emerges more rapidly in models pretrained with music than environmental sounds. Overall, our study provides an operational framework to clarify sufficient built-in and acquired principles that model humans advanced capacity to detect algebraic structures in sounds. Significance StatementExperimentalists have repeatedly observed a human advantage in the detection of algebraic structures, notably through auditory paradigms. This ability to detect structure is thought to be key to the emergence of complex cognitive operations. Yet, it remains debated if this ability is discovered or innate in the form of a specific mechanism. In this article, the authors show how a model progressively and spontaneously learns to detect auditory structure. The model replicate several experimental findings but only under certain developmental conditions. Notably, exposition to music or environmental sounds, but not speech, is sufficient for the emergence of algebraic structure detection. As a result, this work proposes self-supervised learning as a developmental model of abstract cognitive abilities.

neuroscience↗
bioRxiv2023

The best of two worlds: Decoding and source-reconstructing M/EEG oscillatory activity with a unique model

The application of decoding models to electrophysiological data has become standard practice in neuroscience. The use of such methods on sensor space data can, however, limit the interpretability of the results, since brain sources cannot be readily estimated from the decoding of sensor space responses. Here, we propose a new method that combines the common spatial patterns (CSP) algorithm with beamformer source reconstruction for the decoding of oscillatory activity. We compare this method to sensor and source space decoding and show that it performs equally well as source space decoding with respect to both decoding accuracy and source localization without the extensive computational cost. We confirm our simulation results on a real MEG data set. In conclusion, our proposed method performs as good as source space decoding, is highly interpretable in the spatial domain, and has low computational cost.

neuroscience↗
bioRxiv2023

Dimensionality and ramping: Signatures of sentence integration in the dynamics of brains and deep language models

A sentence is more than the sum of its words: its meaning depends on how they combine with one another. The brain mechanisms underlying such semantic composition remain poorly understood. To shed light on the neural vector code underlying semantic composition, we introduce two hypotheses: First, the intrinsic dimensionality of the space of neural representations should increase as a sentence unfolds, paralleling the growing complexity of its semantic representation, and second, this progressive integration should be reflected in ramping and sentence-final signals. To test these predictions, we designed a dataset of closely matched normal and Jabberwocky sentences (composed of meaningless pseudo words) and displayed them to deep language models and to 11 human participants (5 men and 6 women) monitored with simultaneous magneto-encephalography and intracranial electro-encephalography. In both deep language models and electrophysiological data, we found that representational dimensionality was higher for meaningful sentences than Jabberwocky. Furthermore, multivariate decoding of normal versus Jabberwocky confirmed three dynamic patterns: (i) a phasic pattern following each word, peaking in temporal and parietal areas, (ii) a ramping pattern, characteristic of bilateral inferior and middle frontal gyri, and (iii) a sentence-final pattern in left superior frontal gyrus and right orbitofrontal cortex. These results provide a first glimpse into the neural geometry of semantic integration and constrain the search for a neural code of linguistic composition. Significance statementStarting from general linguistic concepts, we make two sets of predictions in neural signals evoked by reading multi-word sentences. First, the intrinsic dimensionality of the representation should grow with additional meaningful words. Second, the neural dynamics should exhibit signatures of encoding, maintaining, and resolving semantic composition. We successfully validated these hypotheses in deep Neural Language Models, artificial neural networks trained on text and performing very well on many Natural Language Processing tasks. Then, using a unique combination of magnetoencephalography and intracranial electrodes, we recorded high-resolution brain data from human participants while they read a controlled set of sentences. Time-resolved dimensionality analysis showed increasing dimensionality with meaning, and multivariate decoding allowed us to isolate the three dynamical patterns we had hypothesized.

neuroscience↗
bioRxiv2022

Tracking the online construction of linguistic meaning through negation

Combinatoric linguistic operations underpin human language processes, but how meaning is composed and refined in the mind of the reader is not well understood. We address this puzzle by exploiting the ubiquitous function of negation. We track the online effects of negation ("not") and intensifiers ("really") on the representation of scalar adjectives (e.g., "good") in parametrically designed behavioral and neurophysiological (MEG) experiments. The behavioral data show that participants first interpret negated adjectives as affirmative and later modify their interpretation towards, but never exactly as, the opposite meaning. Decoding analyses of neural activity further reveal significant above chance decoding accuracy for negated adjectives within 600 ms from adjective onset, suggesting that negation does not invert the representation of adjectives (i.e., "not bad" represented as "good"); furthermore, decoding accuracy for negated adjectives is found to be significantly lower than that for affirmative adjectives. Overall, these results suggest that negation mitigates rather than inverts the neural representations of adjectives. This putative suppression mechanism of negation is supported by increased synchronization of beta-band neural activity in sensorimotor areas. The analysis of negation provides a steppingstone to understand how the human brain represents changes of meaning over time.

neuroscience↗
bioRxiv2022

Top-down information flow drives lexical access when listening to continuous speech

Speech is often structurally and semantically ambiguous. Here we study how the human brain uses sentence context to resolve lexical ambiguity. Twenty-one participants listened to spoken narratives while magneto-encephalography (MEG) was recorded. Stories were annotated for grammatical word class (noun, verb, adjective) under two hypothesised sources of information: bottom-up: the most common word class given the words phonology; top-down: the correct word class given the context. We trained a classifier on trials where the hypotheses matched (about 90%) and tested the classifier on trials where they mismatched. The classifier predicted top-down word class labels, and anti-correlated with bottom-up labels. Effects peaked [~]100ms after word onset over mid-frontal MEG sensors. Phonetic information was encoded in parallel, though peaking later ([~]200ms). Our results support that during continuous speech processing, lexical representations are quickly built in a context-sensitive manner. We showcase multivariate analyses for teasing apart subtle representational distinctions from neural time series.

neuroscience↗
bioRxiv2021

GPT-2's activations predict the degree of semantic comprehension in the human brain

Language transformers, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of deep translation, summarization and dialogue algorithms. However, whether these models encode information that relates to human comprehension remains controversial. Here, we show that the representations of GPT-2 not only map onto the brain responses to spoken stories, but also predict the extent to which subjects understand narratives. To this end, we analyze 101 subjects recorded with functional Magnetic Resonance Imaging while listening to 70 min of short stories. We then fit a linear model to predict brain activity from GPT-2s activations, and correlate this mapping with subjects comprehension scores as assessed for each story. The results show that GPT-2s brain predictions significantly correlate with semantic comprehension. These effects are bilaterally distributed in the language network and peak with a correlation of R=0.50 in the angular gyrus. Overall, this study paves the way to model narrative comprehension in the brain through the lens of modern language algorithms.

neuroscience↗
bioRxiv2021

Spontaneous neural oscillations influence behavior and sensory representations by suppressing neuronal excitability

The ability to process and respond to external input is critical for adaptive behavior. Why, then, do neural and behavioral responses vary across repeated presentations of the same sensory input? Spontaneous fluctuations of neuronal excitability are currently hypothesized to underlie the trial-by-trial variability in sensory processing. To test this, we capitalized on invasive electrophysiology in neurosurgical patients performing an auditory discrimination task with visual cues: specifically, we examined the interaction between prestimulus alpha oscillations, excitability, task performance, and decoded neural stimulus representations. We found that strong prestimulus oscillations in the alpha+ band (i.e., alpha and neighboring frequencies), rather than the aperiodic signal, correlated with a low excitability state, indexed by reduced broadband high-frequency activity. This state was related to slower reaction times and reduced neural stimulus encoding strength. We propose that the alpha+ rhythm modulates excitability, thereby resulting in variability in behavior and sensory representations despite identical input.

neuroscience↗
bioRxiv2020

Neural dynamics of phoneme sequencing in real speech jointly encode order and invariant content

Speech consists of a continuously-varying acoustic signal. Yet human listeners experience it as a sequence of discrete units (phonemes and words). To identify how the brain continuously represents and updates speech units, we recorded two-hour magnetoencephalograms from 21 subjects listening to short narratives. Our encoding and decoding analyses show that the human brain uses a dynamic coding scheme to represent approximately three successive phonetic features at each time sample. These dynamic phonetic representations, not directly accessible from the acoustic signal, are active earlier when phonemes are more predictable, and are sustained when lexical identity is uncertain. The results reveal how phoneme sequences in natural speech are neurally represented and shed new light on how the human brain combines them to probe the mental lexicon. One sentence summaryThe human brain keeps track of the relative order of speech sound sequences by jointly encoding content and elapsed processing time

neuroscience↗
bioRxiv2020

Discriminating the Influence of Correlated Factors from Multivariate Observations: the Back-to-Back Regression

Identifying causes solely from observations can be particularly challenging when i) potential factors are difficult to manipulate independently and ii) observations are multi-dimensional. To address this issue, we introduce "Back-to-Back" regression (B2B), a linear method designed to efficiently measure, from a set of correlated factors, those that most plausibly account for multidimensional observations. First, we prove the consistency of B2B, its links to other linear approaches, and show how it provides a robust, unbiased and interpretable scalar estimate for each factor. Second, we use a variety of simulated data to show that B2B outperforms least-squares regression and cross-decomposition techniques (e.g. canonical correlation analysis and partial least squares) on causal identification when the factors and the observations are partially collinear. Finally, we apply B2B to magneto-encephalography of 102 subjects recorded during a reading task to test whether our method appropriately disentangles the respective contribution of word length and word frequency - two correlated factors known to cause early and late brain responses respectively. The results show that these two factors are better disentangled with B2B than with other standard techniques.

neuroscience↗
bioRxiv2019

The Human Brain encodes a Chronicle of Visual Events at each Instant of Time

The human brain continuously processes streams of visual input. Yet, a single image typically triggers neural responses that extend beyond one second. To understand how such a slow computation copes with real-time processing, we recorded subjects electrical brain activity, while they watched ~5,000 rapidly-changing images. First, we show that each image can be decoded from brain activity for ~1 sec, and demonstrate that the brain simultaneously represents multiple images at each time instant. Second, dynamical system modeling reveals that these sustained representations can be explained by a specific chain of neural circuits, which consist of (i) a hidden maintenance mechanism, and (ii) an observable update mechanism. Third, this neural architecture is localized along the expected visual pathways. Finally, we show that the propagation of low-level representations across the visual hierarchy is a principle shared with deep convolutional networks. Together, these findings provide a general neural mechanism to simultaneously represent successive visual events. SignificanceOur retina are continuously bombarded with a rich flux of visual input. How our brain continuously processes such visual stream is a major challenge to neuroscience. Here, we developed techniques to decode and track, from human brain activity, multiple images flashed in rapid succession. Our results show that the brain simultaneously represents multiple successive images at each time instant. A hierarchy of neural assemblies which continuously propagate multiple visual contents explains our findings. Overall, this study sheds new light on the biological basis of our visual experience.

neuroscience↗
bioRxiv2019

Recurrent Processes Emulate a Cascade of Hierarchical Decisions: Evidence from Spatio-Temporal Decoding of Human Brain Activity

Mounting evidence suggests that perception depends on a largely-feedforward brain network. However, the discrepancy between (i) the latency of the corresponding feedforward responses (150-200 ms) and (ii) the time it takes human subjects to recognize brief images (often >500 ms) suggests that recurrent neuronal activity is critical to visual processing. Here, we use magneto-encephalography to localize, track and decode the feedforward and recurrent responses elicited by brief presentations of variably-ambiguous letters and digits. We first confirm that these stimuli trigger, within the first 200 ms, a feedforward response in the ventral and dorsal cortical pathways. The subsequent activity is distributed across temporal, parietal and prefrontal cortices and leads to a slow and incremental cascade of representations culminating in action-specific motor signals. We introduce an analytical framework to show that these brain responses are best accounted for by a hierarchy of recurrent neural assemblies. An accumulation of computational delays across specific processing stages explains subjects reaction times. Finally, the slow convergence of neural representations towards perceptual categories is quickly followed by all-or-none motor decision signals. Together, these results show how recurrent processes generate, over extended time periods, a cascade of hierarchical decisions that ultimately predicts subjects perceptual reports.

neuroscience↗
bioRxiv2019

Slow cortico-cortical connectivity (2-5Hz) is a new robust signature of conscious states

While long-range cortico-cortical functional connectivity has been reported by several studies as a necessary condition of conscious state, precise empirical evidence is still scarce. In the present work we provide such a direct and conclusive evidence in a set of three experiments. In the two first experiments intracranial-EEG was recorded during four distinct states in the same individuals: conscious wakefulness (CW), rapid-eye-movement sleep (REM), stable periods of slow-wave sleep (SWS) and deep propofol anaesthesia (PA). We discovered that long-range FC, computed by the weighted Symbolic-Mutual-Information (wSMI) in the 2-5Hz frequency band was a specific marker of conscious states that could discriminate CW and REM from SWS and PA. In the third experiment, we generalized this original finding on a large cohort of brain-injured patients by revealing that wSMI in the 2-5 Hz range was also able to accurately discriminate patients in the vegetative state (or unresponsive wakefulness syndrome) from patients in the minimally conscious state. Taken together the present results suggest that 2-5Hz FC is a new and robust signature of conscious states.

neuroscience↗
bioRxiv2018

Differential brain mechanisms of selection and maintenance of information during working memory

Working memory is our ability to select and temporarily hold information as needed for complex cognitive operations. The temporal dynamics of sustained and transient neural activity supporting the selection and holding of memory content is not known. To address this problem, we recorded magnetoencephalography (MEG) in healthy participants performing a retro-cue working memory task in which the selection rule and the memory content varied independently. Multivariate decoding and source analyses showed that selecting the memory content relies on prefrontal and parieto-occipital persistent oscillatory neural activity. By contrast, the memory content was reactivated in a distributed occipito-temporal posterior network, preceding the working memory decision and in a different format that during the visual encoding. These results identify a neural signature of content selection and characterize differentiated spatiotemporal constraints for subprocesses of working memory.

neuroscience↗
bioRxiv2017

Tracking the Canonical Computations of Perceptual Processing in a Quasi-Continuous Sensory Flow

The canonical computations involved in sensory processing, such as neural adaptation and prediction-error signals, have mainly derived from studies investigating the neural responses elicited by a single stimulus. Here, we test whether these computations can be tracked in a quasi-continuous flow of visual stimulation, by correlating scalp electroencephalography (EEG) recordings to simulations of neuronal populations. Fifteen subjects were presented with ~5,000 visual gratings presented in rapid sequences. Our results show that we can simultaneously decode, from the EEG sensors, up to 4 visual stimuli presented sequentially. Temporal generalization and source analyses reveal that the information contained in each stimulus is processed by a \"visual pipeline\": a long cascade of transient processing stages, which can overall encode multiple stimuli at once. Importantly, our data suggest that the early feedforward activity but not the late feedback responses are marked by an adaptation phenomenon. Overall, our approach demonstrates how theoretically-derived computations, as isolated in single-stimulus paradigms, can be generalized to conditions of a continuous flow of sensory information.

neuroscience↗
bioRxiv2017

Riemannian Geometry Boosts Representational Similarity Analyses of Dense Neural Time Series

Representational similarity analysis (RSA) is a popular technique to estimate the structure of mental representations from neuroimaging data. However, RSA can be difficult to estimate for neural time series, where mental representations may be distributed in a highly dimensional space. Here, we show that RSA can be efficiently estimated from dense neural time series using Riemannian geometry. Using a public magneto-encephalography dataset, we decoded 24 classes from the brain evoked responses to 720 visual stimuli. RSA estimated from the confusion matrices of a standard regularized logistic regression achieved an average decoding accuracy of 23% (chance=4%). Our approach based on spatial filtering and Riemannian geometry nearly doubled this score with an average 42% decoding accuracy. Finally, our results revealed how RSA becomes ill-conceived when it derives from confusion matrices of highly accurate multivariate pattern classifications. Instead, we propose to directly estimate RSA from Riemannian metrics without fitting a multivariate pattern classifier. Overall, our approach, based on Riemannian geometry provides a principled and efficient basis to study the structure of mental representations from highly dimensional neural time series.

neuroscience↗
bioRxiv2017

Performance-optimized hierarchical models only partially predict neural responses during perceptual decision making

Models of perceptual decision making have historically been designed to maximally explain behaviour and brain activity independently of their ability to actually perform tasks. More recently, performance-optimized models have been shown to correlate with brain responses to images and thus present a complementary approach to understand perceptual processes. In the present study, we compare how these approaches comparatively account for the spatio-temporal organization of neural responses elicited by ambiguous visual stimuli. Forty-six healthy human subjects performed perceptual decisions on briefly flashed stimuli constructed from ambiguous characters. The stimuli were designed to have 7 orthogonal properties, ranging from low-sensory levels (e.g. spatial location of the stimulus) to conceptual (whether stimulus is a letter or a digit) and task levels (i.e. required hand movement). Magneto-encephalography source and decoding analyses revealed that these 7 levels of representations are sequentially encoded by the cortical hierarchy, and actively maintained until the subject responds. This hierarchy appeared poorly correlated to normative, drift-diffusion, and 5-layer convolutional neural networks (CNN) optimized to accurately categorize alpha-numeric characters, but partially matched the sequence of activations of 3/6 state-of-the-art CNNs trained for natural image labeling (VGG-16, VGG-19, MobileNet). Additionally, we identify several systematic discrepancies between these CNNs and brain activity, revealing the importance of single-trial learning and recurrent processing. Overall, our results strengthen the notion that performance-optimized algorithms can converge towards the computational solution implemented by the human visual system, and open possible avenues to improve artificial perceptual decision making.

neuroscience↗
bioRxiv2016

A theory of working memory without consciousness or sustained activity

Working memory and conscious perception are thought to share similar brain mechanisms, yet recent reports of non-conscious working memory challenge this view. Combining visual masking with magnetoencephalography, we demonstrate the reality of non-conscious working memory and dissect its neural mechanisms. In a spatial delayed-response task, participants reported the location of a subjectively unseen target above chance-level after a long delay. Conscious perception and conscious working memory were characterized by similar signatures: a sustained desynchronization in the alpha/beta band over frontal cortex, and a decodable representation of target location in posterior sensors. During non-conscious working memory, such activity vanished. Our findings contradict models that identify working memory with sustained neural firing, but are compatible with recent proposals of activity-silent working memory. We present a theoretical framework and simulations showing how slowly decaying synaptic changes allow cell assemblies to go dormant during the delay, yet be retrieved above chance-level after several seconds.

neuroscience↗