Biology subjects

Norman-Haignere, S. V.

Publications and source records attributed to Norman-Haignere, S. V..

11 recordsLinked to original sources

bioRxiv2026

Neural prediction decorrelation reveals that adversarial robustness substantially improves DNN prediction accuracy across the entire human auditory cortex

Sensory neuroscientists seek to model the neural computations that encode complex stimuli. Distinct encoding models often make similar predictions for natural stimuli such as speech, posing a challenge for model comparison. We developed a method to synthesize stimuli that decorrelate model predictions across a neural population, termed neural prediction decorrelation (NPD). Using fMRI responses to NPD sounds, we compared standard and adversarially robust deep neural network models of human auditory cortex. Prediction accuracy for NPD sounds was substantially better for the adversarially robust model in every region tested, an effect completely masked with natural sounds. Population responses to natural and synthesized NPD sounds shared an interpretable low-dimensional organization that was reproduced by the robust encoding model. NPD provides a general approach for comparing encoding models and reveals that adversarial robustness expands the predictive power of DNNs beyond natural stimuli, which is likely critical for targeting population activity through stimulus synthesis.

neuroscience↗

bioRxiv2024

Convolutional neural network models describe the encoding subspace of local circuits in auditory cortex

Convolutional neural networks (CNNs) provide powerful models of neural sensory encoding, but their complexity makes it difficult to discern computations that support their performance. To address this limitation, we developed a linear-nonlinear subspace model that identifies the most informative sensory dimensions captured by a CNN. A CNN was trained on single-neuron data recorded from auditory cortex of ferrets during presentation of a large natural sound set. Each neurons linear tuning subspace was computed by applying dimensionality reduction to the gradient of CNN output relative to input. Subspace projections were combined nonlinearly to predict neural activity. The resulting model was functionally equivalent to the CNN. Analysis of trained models showed that responses of local neural populations sparsely tiled a shared stimulus subspace. Encoding properties also differed between cell types and layers, reflecting their position in the cortical circuit. More generally, these results establish a framework for interpreting deep learning-based encoding models. Significance statementAuditory cortex mediates the representation and discrimination of complex sound features. Many models have been proposed for cortical sound encoding, varying in their generality, interpretability, and ease of fitting. It has been difficult to determine if and what different functional properties are captured by different models. This study shows that two families of encoding models, convolutional neural networks (CNNs) and tuning subspace models, account for the same functional properties, providing an important analytical link between accurate models that are easy to fit (CNNs) and models that are straightforward to interpret (tuning subspace).

neuroscience↗

bioRxiv2024

Temporal integration in human auditory cortex is predominantly yoked to absolute time, not structure duration

Sound structures such as phonemes and words have highly variable durations. Thus, there is a fundamental difference between integrating across absolute time (e.g., 100 ms) vs. sound structure (e.g., phonemes). Auditory and cognitive models have traditionally cast neural integration in terms of time and structure, respectively, but the extent to which cortical computations reflect time or structure remains unknown. To answer this question, we rescaled the duration of all speech structures using time stretching/compression and measured integration windows in the human auditory cortex using a new experimental/computational method applied to spatiotemporally precise intracranial recordings. We observed significantly longer integration windows for stretched speech, but this lengthening was very small ([~]5%) relative to the change in structure durations, even in non-primary regions strongly implicated in speech-specific processing. These findings demonstrate that time-yoked computations dominate throughout the human auditory cortex, placing important constraints on neurocomputational models of structure processing.

neuroscience↗

bioRxiv2024

Dynamic modeling of EEG responses to natural speech reveals earlier processing of predictable words

In recent years, it has become clear that EEG indexes the comprehension of natural, narrative speech. One particularly compelling demonstration of this fact can be seen by regressing EEG responses to speech against measures of how individual words in that speech linguistically relate to their preceding context. This approach produces a so-called temporal response function that displays a centro-parietal negativity reminiscent of the classic N400 component of the event-related potential. One shortcoming of previous implementations of this approach is that they have typically assumed a linear, time-invariant relationship between the linguistic speech features and the EEG responses. In other words, the analysis typically assumes that the response has the same shape and timing for every word - and only varies (linearly) in terms of its amplitude. In the present work, we relax this assumption under the hypothesis that responses to individual words may be processed more rapidly when they are predictable. Specifically, we introduce a framework wherein the standard linear temporal response function can be modulated in terms of its amplitude, latency, and temporal scale based on the expectedness of the current and prior words. We use the proposed approach to model EEG recorded from a set of participants who listened to an audiobook narrated by a single talker, and a separate set of participants who attended to one of two concurrently presented audiobooks. We show that expected words are processed faster - evoking lower amplitude N400-like responses with earlier peaks - and that this effect is driven both by the words own predictability and the predictability of the immediately preceding word. Additional analysis suggests that this finding is not simply explained based on how quickly words can be disambiguated from their phonetic neighbors. As such, our study demonstrates that the timing and amplitude of brain responses to words in natural speech depend on their predictability. By accounting for these effects, our framework also improves the accuracy with which neural responses to natural speech can be modeled.

neuroscience↗

bioRxiv2023

Preserved functional organization of human auditory cortex in individuals missing one temporal lobe from infancy

Human cortical responses to natural sounds, measured with fMRI, can be approximated as the weighted sum of a small number of canonical response patterns (components), each having interpretable functional and anatomical properties. Here, we asked whether this organization is preserved in cases where only one temporal lobe is available due to early brain damage by investigating a unique family: one sibling born without a left temporal lobe, another without a right temporal lobe, and a third anatomically neurotypical. We analyzed fMRI responses to diverse natural sounds within the intact hemispheres of these individuals and compared them to 12 neurotypical participants. All siblings manifested the neurotypical auditory responses in their intact hemispheres. These results suggest that the development of the auditory cortex in each hemisphere does not depend on the existence of the other hemisphere, highlighting the redundancy and equipotentiality of the bilateral auditory system.

neuroscience↗

bioRxiv2021

Music-selective cortex is sensitive to structure in both pitch and time

Music is distinguished from other natural sounds by the presence of relatively discrete notes, which are then organized across pitch and time to convey melody, harmony, and rhythm. Growing evidence suggests that small clusters of neural populations within anterior and posterior human non-primary auditory cortex respond selectively to musical structure. However, it is unclear whether this selectivity reflects short-term musical structure at the level of individual notes, and/or the patterning of notes in pitch and time. We used fMRI voxel decomposition to measure the response of music-selective and non-selective auditory neural populations to synthetic music and drum stimuli whose notes were scrambled in pitch and/or time, disrupting musical pattern structure while largely preserving note-level structure. We observed reliably stronger responses to music with intact pitch and temporal pattern structure in both anterior and posterior music-selective regions bilaterally, but little difference between intact and scrambled music in non-selective populations. Further, only music-selective populations showed reliably stronger responses to note-scrambled music compared with non-music sounds. These results suggest that musical structure involving both individual notes and their patterning over time is specifically represented in localized music-selective neural populations of human non-primary auditory cortex.

neuroscience↗

bioRxiv2020

Hierarchical integration across multiple timescales in human auditory cortex

To derive meaning from sound, the brain must integrate information across tens (e.g. phonemes) to hundreds (e.g. words) of milliseconds, but the neural computations that enable multiscale integration remain unclear. Prior evidence suggests that human auditory cortex analyzes sound using both generic acoustic features (e.g. spectrotemporal modulation) and category-specific computations, but how these putatively distinct computations integrate temporal information is unknown. To answer this question, we developed a novel method to estimate neural integration periods and applied the method to intracranial recordings from human epilepsy patients. We show that integration periods increase three-fold as one ascends the auditory cortical hierarchy. Moreover, we find that electrodes with short integration periods (~50-150 ms) respond selectively to spectrotemporal modulations, while electrodes with long integration periods (~200-300 ms) show prominent selectivity for sound categories such as speech and music. These findings reveal how multiscale temporal analysis organizes hierarchical computation in human auditory cortex.

neuroscience↗

bioRxiv2020

Distinct higher-order representations of natural sounds in human and ferret auditory cortex

Little is known about how neural representations of natural sounds differ across species. For example, speech and music play a unique role in human hearing, yet it is unclear how auditory representations of speech and music differ between humans and other animals. Using functional Ultrasound imaging, we measured responses in ferrets to a set of natural and spectrotemporally-matched synthetic sounds previously tested in humans. Ferrets showed similar lower-level frequency and modulation tuning to that observed in humans. But while humans showed prominent selectivity for natural vs. synthetic speech and music in non-primary regions, ferret responses to natural and synthetic sounds were closely matched throughout primary and non-primary auditory cortex, even when tested with ferret vocalizations. This finding reveals that auditory representations in humans and ferrets diverge sharply at late stages of cortical processing, potentially driven by higher-order processing demands in speech and music.

neuroscience↗

bioRxiv2020

Causal inference in environmental sound recognition

Sound is caused by physical events in the world. Do humans infer these causes when recognizing sound sources? We tested whether the recognition of common environmental sounds depends on the inference of a basic physical variable - the source intensity (i.e., the power that produces a sound). A sources intensity can be inferred from the intensity it produces at the ear and its distance, which is normally conveyed by reverberation. Listeners could thus use intensity at the ear and reverberation to constrain recognition by inferring the underlying source intensity. Alternatively, listeners might separate these acoustic cues from their representation of a sounds identity in the interest of invariant recognition. We compared these two hypotheses by measuring recognition accuracy for sounds with typically low or high source intensity (e.g., pepper grinders vs. trucks) that were presented across a range of intensities at the ear or with reverberation cues to distance. The recognition of low-intensity sources (e.g., pepper grinders) was impaired by high presentation intensities or reverberation that conveyed distance, either of which imply high source intensity. Neither effect occurred for high-intensity sources. The results suggest that listeners implicitly use the intensity at the ear along with distance cues to infer a sources power and constrain its identity. The recognition of real-world sounds thus appears to depend upon the inference of their physical generative parameters, even generative parameters whose cues might otherwise be separated from the representation of a sounds identity.

animal behavior and cognition↗

bioRxiv2020

Cortical music selectivity does not require musical training

Recent work has shown that human auditory cortex contains neural populations anterior and posterior to primary auditory cortex that respond selectively to music. However, it is unknown how this selectivity for music arises. To test whether musical training is necessary, we measured fMRI responses to 192 natural sounds in 10 people with almost no musical training. When voxel responses were decomposed into underlying components, this group exhibited a music-selective component that was very similar in response profile and anatomical distribution to that previously seen in individuals with moderate musical training. We also found that musical genres that were less familiar to our participants (e.g., Balinese gamelan) produced strong responses within the music component, as did drum clips with rhythm but little melody, suggesting that these neural populations are broadly responsive to music as a whole. Our findings demonstrate that the signature properties of neural music selectivity do not require musical training to develop, showing that the music-selective neural populations are a fundamental and widespread property of the human brain. NEW & NOTEWORTHYWe show that music-selective neural populations are clearly present in people without musical training, demonstrating that they are a fundamental and widespread property of the human brain. Additionally, we show music-selective neural populations respond strongly to music from unfamiliar genres as well as music with rhythm but little pitch information, suggesting that they are broadly responsive to music as a whole.

neuroscience↗

bioRxiv2018

Neural responses to natural and model-matched stimuli reveal distinct computations in primary and non-primary auditory cortex

A central goal of sensory neuroscience is to construct models that can explain neural responses to complex, natural stimuli. As a consequence, sensory models are often tested by comparing neural responses to natural stimuli with model responses to those stimuli. One challenge is that distinct model features are often correlated across natural stimuli, and thus model features can predict neural responses even if they do not in fact drive them. Here we propose a simple alternative for testing a sensory model: we synthesize stimuli that yield the same model response as a natural stimulus, and test whether the natural and \"model-matched\" stimulus elicit the same neural response. We used this approach to test whether a common model of auditory cortex - in which spectrogram-like peripheral input is processed by linear spectrotemporal filters - can explain fMRI responses in humans to natural sounds. Prior studies have that shown that this model has good predictive power throughout auditory cortex, but this finding could reflect stimulus-driven correlations. We observed that fMRI voxel responses to natural and model-matched stimuli were nearly equivalent in primary auditory cortex, but that non-primary regions showed highly divergent responses to the two sound sets, suggesting that neurons in non-primary regions extract higher-order properties not made explicit by traditional models. This dissociation between primary and non-primary regions was not clear from model predictions due to the influence of stimulus-driven response correlations. Our methodology enables stronger tests of sensory models and could be broadly applied in other domains.\n\nAuthor SummaryModeling neural responses to natural stimuli is a core goal of sensory neuroscience. Here we propose a new approach for testing sensory models: we synthesize a \"model-matched\" stimulus that yields the same model response as a natural stimulus, and test whether it produces the same neural response. We used model-matching to test whether a standard model of auditory cortex can explain human cortical responses measured with fMRI. Model-matched stimuli produced nearly equivalent voxel responses in primary auditory cortex, but highly divergent responses in non-primary regions. This dissociation was not evident using more standard approaches for model testing, and suggests that non-primary regions compute higher-order stimulus properties not captured by traditional models. The methodology could be broadly applied in other domains.

neuroscience↗