Biology subjects

Norcia, A. M.

Publications and source records attributed to Norcia, A. M..

32 records · Page 2Linked to original sources

bioRxiv2021

Increasing neural network robustness improves match to macaque V1 eigenspectrum, spatial frequency preference and predictivity

Task-optimized convolutional neural networks (CNNs) show striking similarities to the ventral visual stream. However, human-imperceptible image perturbations can cause a CNN to make incorrect predictions. Here we provide insight into this brittleness by investigating the representations of models that are either robust or not robust to image perturbations. Theory suggests that the robustness of a system to these perturbations could be related to the power law exponent of the eigenspectrum of its set of neural responses, where power law exponents closer to and larger than one would indicate a system that is less susceptible to input perturbations. We show that neural responses in mouse and macaque primary visual cortex (V1) obey the predictions of this theory, where their eigenspectra have power law exponents of at least one. We also find that the eigenspectra of model representations decay slowly relative to those observed in neurophysiology and that robust models have eigenspectra that decay slightly faster and have higher power law exponents than those of non-robust models. The slow decay of the eigenspectra suggests that substantial variance in the model responses is related to the encoding of fine stimulus features. We therefore investigated the spatial frequency tuning of artificial neurons and found that a large proportion of them preferred high spatial frequencies and that robust models had preferred spatial frequency distributions more aligned with the measured spatial frequency distribution of macaque V1 cells. Furthermore, robust models were quantitatively better models of V1 than non-robust models. Our results are consistent with other findings that there is a misalignment between human and machine perception. They also suggest that it may be useful to penalize slow-decaying eigenspectra or to bias models to extract features of lower spatial frequencies during task-optimization in order to improve robustness and V1 neural response predictivity. Author summaryConvolutional neural networks (CNNs) are the most quantitatively accurate models of multiple visual areas. In contrast to humans, however, their image classification behaviour can be modified drastically by human-imperceptible image perturbations. To provide insight as to why CNNs are so brittle, we investigated the image features extracted by models that are robust and not robust to these image perturbations. We found that CNNs had a preference for high spatial frequency image features, unlike primary visual cortex (V1) cells. Models that were more robust to image perturbations had a preference for image features more aligned with those extracted by V1 and also improved predictions of neural responses in V1. This suggests that the dependence on high-frequency image features for image classification may be related to the image perturbations affecting models but not humans. Our work is consistent with other findings that CNNs may be relying on image features not aligned with those used by humans for image classification and suggests possible optimization targets to improve the robustness of and the V1 correspondence of CNNs.

neuroscience↗

bioRxiv2021

Unsupervised Models of Mouse Visual Cortex

Studies of the mouse visual system have revealed a variety of visual brain areas that are thought to support a multitude of behavioral capacities, ranging from stimulus-reward associations, to goal-directed navigation, and object-centric discriminations. However, an overall understanding of the mouses visual cortex, and how it supports a range of behaviors, remains unknown. Here, we take a computational approach to help address these questions, providing a high-fidelity quantitative model of mouse visual cortex and identifying key structural and functional principles underlying that models success. Structurally, we find that a comparatively shallow network structure with a low-resolution input is optimal for modeling mouse visual cortex. Our main finding is functional - that models trained with task-agnostic, self-supervised objective functions based on the concept of contrastive embeddings are much better matches to mouse cortex, than models trained on supervised objectives or alternative self-supervised methods. This result is very much unlike in primates where prior work showed that the two were roughly equivalent, naturally leading us to ask the question of why these self-supervised objectives are better matches than supervised ones in mouse. To this end, we show that the self-supervised, contrastive objective builds a general-purpose visual representation that enables the system to achieve better transfer on out-of-distribution visual scene understanding and reward-based navigation tasks. Our results suggest that mouse visual cortex is a low-resolution, shallow network that makes best use of the mouses limited resources to create a light-weight, general-purpose visual system - in contrast to the deep, high-resolution, and more categorization-dominated visual system of primates. Author summaryStudies of mouse visual behavior have revealed a multitude of visual abilities, ranging from stimulus-reward associations, to goal-directed navigation, and object-centric discriminations. A principled system-wide model of mouse visual cortex would be useful both in providing an organizing theory for this wide spectrum of behaviors, and enabling practical technology for many model-driven studies of mouse neuroscience more broadly. However, the standard approach to creating quantitatively accurate models of primate visual cortex has been less successful with the mouse system. Here we identify critical computational features needed to capture mouse-specific neural stimulus-response patterns, and illustrate how these features can be interpreted as giving the highly resource-limited mouse brain a comparative advantage in self-learning a task-general visual representation.

neuroscience↗

bioRxiv2021

Inter-Subject EEG Correlation Reflects Time-Varying Engagement with Natural Music

Musical engagement can be conceptualized through various activities, modes of listening, and listener states--among these a state of focused engagement. Recent research has reported that this state can be indexed by the inter-subject correlation (ISC) of EEG responses to a shared naturalistic stimulus. While statistically significant ISC has been reported during music listening, these reports have considered only correlations computed across entire excerpts and do not provide insights into time-varying engagement. Here we present the first EEG-ISC investigation of time-varying engagement within a musical work. From a sample of 23 adult musicians who listened to a cello concerto movement, we find varying levels of ISC throughout the excerpt. In particular, significant ISC is observed during periods of musical tension that build to climactic highpoints, but not at the highpoints themselves. In addition, we find that a control stimulus retaining envelope characteristics of the intact music, but little other temporal structure, also elicits significant neural correlation, though to a lesser extent than the original excerpt. In all, our findings shed light on temporal dynamics of listener engagement during music listening, establish connections between salient musical events and EEG ISC, and clarify specific listener states that are indexed by this measure.

neuroscience↗

bioRxiv2021

Neural Sources Underlying Visual Word Form Processing as Revealed by Steady State Visual Evoked Potentials (SSVEP)

EEG has been central to investigations of the time course of various neural functions underpinning visual word recognition. Recently the steady-state visual evoked potential (SSVEP) paradigm has been increasingly adopted for word recognition studies due to its high signal-to-noise ratio. Such studies, however, have been typically framed around a single source in the left ventral occipitotemporal cortex (vOT). Here, we combine SSVEP recorded from 16 adult native English speakers with a data-driven spatial filtering approach--Reliable Components Analysis (RCA)--to elucidate distinct functional sources with overlapping yet separable time courses and topographies that emerge when contrasting words with pseudofont visual controls. The first component topography was maximal over left vOT regions with an early latency (approximately 180 msec). A second component was maximal over more dorsal parietal regions with a longer latency (approximately 260 msec). Both components consistently emerged across a range of parameter manipulations including changes in the spatial overlap between successive stimuli, and changes in both base and deviation frequency. We then contrasted word-in-nonword and word-in-pseudoword to test the hierarchical processing mechanisms underlying visual word recognition. Results suggest that these hierarchical contrasts fail to evoke a unitary component that might be reasonably associated with lexical access.

neuroscience↗

bioRxiv2021

Dynamics of absolute and relative disparity processing in human visual cortex

Cortical processing of binocular disparity is believed to begin in V1 where cells are sensitive to absolute disparity, followed by the extraction of relative disparity in higher visual areas. While much is known about the cortical distribution and spatial tuning of disparity-selective neurons, the relationship between their spatial and temporal properties is less well understood. Here, we use steady-state Visual Evoked Potentials and dynamic random dot stereograms to characterize the temporal dynamics of spatial mechanisms in human visual cortex that are primarily sensitive to either absolute or relative disparity. Stereograms alternated between disparate and non-disparate states at 2 Hz. By varying the spatial frequency content of the disparate fields from a planar surface to corrugated ones, we biased responses towards absolute vs. relative disparities. Reliable Components Analysis was used to derive two dominant sources from the 128 channel EEG records. The first component (RC1) was maximal over the occipital pole while the second component (RC2) was maximal over right lateral occipital electrodes. In RC1, first harmonic responses were sustained, tuned for corrugation frequency, and sensitive to the presence of disparity references, consistent with prior psychophysical sensitivity measurements. By contrast, the second harmonic, associated with transient processing, was not spatially tuned and was indifferent to references, consistent with it being generated by an absolute disparity mechanism. In RC2, the sustained response component showed similar tuning and sensitivity to references. However, sensitivity for absolute disparity dropped off, and transient signals were mainly driven by the lowest corrugation frequencies. SIGNIFICANCE STATEMENTSustained and transient mechanisms have been demonstrated across sensory systems and may reflect a common coding strategy. This strategy may be useful for the analysis of the form and duration of complex events in the case of sustained signals, and the onset and location of events in the case of transient signals. Here, we provide direct neural correlates of sustained and transient disparity mechanisms in human visual cortex. Early visual cortex is sensitive to both relative and absolute disparity, with the former being processed in a sustained fashion and the latter in a more transient fashion. Outside of early visual cortex, sustained relative disparity responses are readily measurable, but transient responses and responses to absolute disparity are not.

neuroscience↗

bioRxiv2020

PARIETAL CONTRIBUTIONS TO ABSTRACT NUMEROSITY MEASURED WITH STEADY STATE VISUAL EVOKED POTENTIALS

Non-symbolic number changes produce transient Event Related Potentials over parietal electrodes, while numerosity effects measured with Steady-State Visual Evoked Potentials (SSVEPs) appear to originate in occipital cortex. We hypothesized that the stimulation rates used in previous SSVEP studies may be too rapid to drive parietal numerosity mechanisms. Here we recorded SSVEPs and behavioral reports over a slower range of temporal frequencies than previously used. Isoluminant dot stimuli updated at a consistent "carrier" frequency (3-6 Hz) while periodic changes in numerosity (e.g. 8[->]5) formed an even slower "oddball" frequency (0.5-1 Hz). Each numerosity oddball condition had a matched control condition where the number of dots did not change. Carrier frequencies induced SSVEPs with midline occipital topographies that did not differentiate the presence or absence of numerosity oddballs. By contrast, SSVEPs at oddball frequencies had parietal topographies and responded more strongly when oddballs were present. Consistent with our hypothesis, numerosity effects were stronger at slower stimulation rates. In a second study, the numerosity change was either supra-threshold (e.g. 8[->]5 dots) or near the threshold required for detecting numerosity changes (e.g. 8[->]9 dots). We found robust parietal responses for the supra-threshold case only, indicating a numerical distance effect. A third study replicated the parietal oddball SSVEP effect across four distinct suprathreshold numerosity-change conditions and showed that number change direction does not influence the effect. These findings show that SSVEP oddball paradigms can probe parietal computations of abstract numerosity, and may provide a rapid, portable approach to quantifying number sense within educational settings.

neuroscience↗

bioRxiv2019

Time-Resolved Correspondences Between Deep Neural Network Layers and EEG Measurements in Object Processing

The ventral visual stream is known to be organized hierarchically, where early visual areas processing simplistic features feed into higher visual areas processing more complex features. Hierarchical convolutional neural networks (CNNs) were largely inspired by this type of brain organization and have been successfully used to model neural responses in different areas of the visual system. In this work, we aim to understand how an instance of these models corresponds to temporal dynamics of human object processing. Using representational similarity analysis (RSA) and various similarity metrics, we compare the model representations with two electroencephalography (EEG) data sets containing responses to a shared set of 72 images. We find that there is a hierarchical relationship between the depth of a layer and the time at which peak correlation with the brain response occurs for certain similarity metrics in both data sets. However, when comparing across layers in the neural network, the correlation onset time did not appear in a strictly hierarchical fashion. We present two additional methods that improve upon the achieved correlations by optimally weighting features from the CNN and show that depending on the similarity metric, deeper layers of the CNN provide a better correspondence than shallow layers to later time points in the EEG responses. However, we do not find that shallow layers provide better correspondences than those of deeper layers to early time points, an observation that violates the hierarchy and is in agreement with the finding from the onset-time analysis. This work makes a first comparison of various response features--including multiple similarity metrics and data sets--with respect to a neural network.

neuroscience↗

bioRxiv2019

Natural Music Evokes Correlated EEG Responses Reflecting Temporal Structure and Beat

The brain activity of multiple subjects has been shown to synchronize during salient moments of natural stimuli, suggesting that correlation of neural responses indexes a brain state operationally termed engagement. While past electroencephalography (EEG) studies have considered both auditory and visual stimuli, the extent to which these results generalize to music--a temporally structured stimulus for which the brain has evolved specialized circuitry--is less understood. Here we investigated neural correlation during natural music listening by recording dense-array EEG responses from N = 48 adult listeners as they heard real-world musical works, some of which were temporally disrupted through shuffling of short-term segments (measures), reversal, or randomization of phase spectra. We measured neural correlation across responses (inter-subject correlation) and between responses and stimulus envelope fluctuations (stimulus-response correlation) in the time and frequency domains. Stimuli retaining basic musical features evoked significantly correlated neural responses in all analyses. However, while unedited songs were self-reported as most pleasant, time-domain correlations were highest during measure-shuffled versions. Frequency-domain measures of correlation (coherence) peaked at frequencies related to the musical beat, although the magnitudes of these spectral peaks did not explain the observed temporal correlations. Our findings show that natural music evokes significant inter-subject and stimulus-response correlations, and suggest that the neural correlates of musical engagement may be distinct from those of enjoyment.

neuroscience↗

bioRxiv2018

Dynamics of contrast decrement and increment responses in human visual cortex

The goal of the present experiments was to determine whether electrophysiological response properties of the ON and OFF visual pathways observed in animal experimental models can be observed in human. Visual Evoked Potentials (VEPs) were recorded in response to equivalent magnitude contrast increments and decrements presented using sawtooth temporal waveforms at a temporal frequency of 2.73 Hz. VEP response waveforms and response spectra for incremental and decremental stimuli were analyzed as a function of stimulus size and visual field location in 68 healthy adult participants. VEP response were larger in amplitude and shorter in latency for contrast decrements than for contrast increments. The spatial tuning was narrower for contrast decrements than for contrast increments and responses were larger for displays that were scaled for cortical magnification. VEPs recorded at the scalp differ between contrast decrements and increments of equal Weber contrast in a fashion that parallels results from the early visual system of cats and monkeys. Because our assay allows differential detection of ON and OFF pathway activity in human, the approach may be useful in future work on disease detection and treatment monitoring.

neuroscience↗

bioRxiv2018

Autism sensory dysfunction in an evolutionarily conserved system

There is increasing evidence for a strong genetic basis for autism, with many genetic models being developed in an attempt to replicate autistic symptoms in animals. However, current animal behaviour paradigms rarely match the social and cognitive behaviours exhibited by autistic individuals. Here we instead assay another functional domain - sensory processing - known to be affected in autism to test a novel genetic autism model in Drosophila melanogaster. We show similar visual response alterations and a similar development trajectory in Nhe3 mutant flies (total N=72) and in autistic human participants (total N=154). We report a dissociation between first- and second-order electrophysiological visual responses to steady-state stimulation in adult mutant fruit flies that is strikingly similar to the response pattern in human adults with ASD as well as that of a large sample of neurotypical individuals with high numbers of autistic traits. We explain this as a genetically driven, selective signalling alteration in transient visual dynamics. In contrast to adults, autistic children show a decrease in the first-order response that is matched by the fruit fly model, suggesting that a compensatory change in processing occurs during development. Our results provide the first animal model of autism comprising a differential developmental phenotype in visual processing.

neuroscience↗

bioRxiv2018

Functional organization of motion and disparity sensitivity in human visual cortex

Vision with two eyes makes perception of weak visual contrast signals easier and, due to the lateral separation of the eyes, allows for the triangulation of depth relationships. While binocular summation of contrast signals affords the observer increased sensitivity, binocular summation of spatial cues related to changes in depth is associated with decreased sensitivity to the corresponding retinal image displacements. Perceptual models of contrast and motion-in-depth sensitivity have explained this divergence in sensitivity by proposing that probabilistic neural noise limits summing and differencing operations on small signals. Because these models do not scale well for highly suprathreshold visual signals typical of the natural environment, we approached the question of how dynamic binocular image differences are coded using direct neural measurements. Here we use Steady-State Visual Evoked Potentials in human participants to show that inter-ocular differences in retinal image motion that produce elevated perceptual thresholds generate strongly suppressed evoked response amplitudes compared to motion that is matched between the two eyes. This suppression is strongly dependent on the availability of well-defined spatial references in the image and is highly immature in 5-month-old infants. Because the suppression is of equal strength for horizontal and vertical directions of motion, it is not specific to the perception of motion in depth. Relational image cues play a critical role in early to intermediate perceptual processing stages, and these results suggest that a succession of spatial and inter-ocular differencing operations condition the visual signal representation, prior to the extraction of motion-in-depth.\n\nSignificance StatementThe present work underscores the importance of relational spatial cues in both the motion and disparity domains for binocular visual coding. Relative motion and relative disparity cues not only support fine-grain displacement sensitivity but strongly influence suprathreshold responsiveness. Extraction of these cues supports a powerful binocular interaction within the motion pathway that is suppressive in nature and poorly developed in infants. This suppressive interaction is present for both horizontal and vertical directions of motion and is thus not specific to motion-in-depth, as previously believed, but is rather hypothesized to be a pre-processing step, with motion-in-depth being computed at a later or separate stage.

neuroscience↗

bioRxiv2017

From image formation to image representation: the human visual system preserves the hierarchy of 2-dimensional pattern regularity

Symmetries are present at many scales in images of natural scenes, due to a complex interplay of physical forces that govern pattern formation in nature. The importance of symmetry for visual perception has been known at least since the gestalt movement of the early 20th century. Since then, symmetry has been shown to contribute to the perception of shapes[1, 2] scenes[3] and surface properties[4], as well as the social process of mate selection[5]. In the two spatial dimensions relevant for images, the four fundamental symmetries, reflection, rotation, translation and glide reflection, can be combined in 17 distinct ways, the \"wallpaper\" groups[6-8]. The 17 wallpaper obey a hierarchy of complexity, determined by mathematical group theory, where simpler groups are subgroups of more complex ones[9]. Here we use Steady-State Visual Evoked Potentials (SSVEPs) to measure responses to the complete set of wallpaper groups, and present a complete description of the neural basis of symmetry. We find that activity in human visual cortex is remarkably consistent with the hierarchical relationships between the wallpaper groups. Specifically, the amplitudes of symmetry-specific responses in individual participants (n=25) preserve these relationships at an above-chance level in 88.3% (53 out of 60) of cases. Visual cortex thus encodes all the fundamental symmetries using a representational structure that closely approximates the subgroup relationships from group theory. Given that most participants had no knowledge of group theory, the ordered structure of visual responses to wallpaper groups is likely learned implicitly from regularities in the visual environment.

neuroscience↗

bioRxiv2017

Image segmentation based on relative motion and relative disparity cues in topographically organized areas of human visual cortex

The borders between objects and their backgrounds create discontinuities in image feature maps that can be used to recover object shape. Here we used functional magnetic resonance imaging (fMRI) to study the sensitivity of visual cortex to two of the most important image segmentation cues: relative motion and relative disparity. Relative motion and disparity cues were isolated using random-dot kinematograms and stereograms, respectively. For motion-defined boundaries, we found a strong retinotopically organized representation of a 2-degree radius motion-defined disk, starting in V1 and extending though V2 and V3. In the surrounding region, we observed phase-inverted activations indicative of suppression, extending out to at least 6 degrees of retinal eccentricity. For relative disparity, figure responses were only robust in V3, while suppression was observed in all early visual areas. When attention was captured at fixation, figure responses persisted while suppression did not, suggesting that suppression is generated by attentional feedback from higher-order visual areas. Outside of the early visual areas, several areas were sensitive to both types of cues, most notably hV4, LO1 and V3B, making them additional candidate areas for motion- and disparity-cue combination. The overall pattern of extra-striate activations is consistent with recent three-stream models of cortical organization.

neuroscience↗

bioRxiv2017

MatClassRSA: A Matlab toolbox for M/EEG classification and visualization of proximity matrices

MatClassRSA is a Matlab toolbox that performs M/EEG classification and produces various visualizations of the resulting confusion matrices. This toolbox is aimed toward cognitive neuroscience researchers who wish to apply classification-style analyses to repeated trials of data. The functionalities of the toolbox fall into three categories: (1) M/EEG classification, (2) computation of Representational Dissimilarity Matrices (RDMs) from confusion or other proximity matrices, and (3) clustering and tree visualizations of RDMs. MatClassRSA combines out-of-the-box functionality with a variety of customization options. Usage of the toolbox requires only a high-level understanding of machine learning and the ability to call Matlab functions from scripts.

neuroscience↗