Biology subjects

Hasson, U.

Publications and source records attributed to Hasson, U..

17 recordsLinked to original sources

bioRxiv2018

Infant and adult brains are coupled to the dynamics of natural communication

Infancy is the foundational period for learning from adults, and the dynamics of the social environment have long been proposed as central to childrens development. Here we reveal a novel, highly naturalistic approach for studying live interactions between infants and adults. Using functional near-infrared spectroscopy (fNIRS), we simultaneously and continuously measured the brains of infants (9-15 months) and an adult while they communicated and played with each other in real time. We found that time-locked neural coupling within dyads was significantly greater when they interacted with each other than with control individuals. In addition, we found that both infant and adult brains continuously tracked the moment-to-moment fluctuations of mutual gaze, infant emotion, and adult speech prosody with high temporal precision. This investigation advances what is currently known about how the brains and behaviors of infants both shape and reflect those of adults during real-life communication.

neuroscience↗

bioRxiv2018

Electrocorticographic responses to time-compressed speech vary across the cortical auditory hierarchy

Human listeners understand spoken language across a variety of rates, but when speech is presented three times or more faster than its usual rate, it becomes unintelligible. How the brain achieves such tolerance and why speech becomes unintelligible above certain rates is still unclear. We addressed these questions using electrocorticography (ECoG) recordings in 7 epileptic patients (two female). Patients rated the intelligibility of sentences presented at the original rate (100%), speeded rates (33% or 66% of the original sentence duration) and a slowed rate (150%). We then examined which parameters of the neural response covary with the transition from intelligible to unintelligible speech. Specifically, we asked whether neural responses: 1) track the acoustic envelope of the incoming speech; 2) \"scale\" with speech rate, i.e. whether neural responses elicited by slowed and speeded sentences can be linearly scaled to match the responses to the original sentence. Behaviorally, intelligibility was at ceiling for speech rates of 66% and above, but dropped significantly for the 33% rate. At the neural level, Superior Temporal Gyrus regions (STG) in close proximity to A1 ( low-level) tracked the acoustic envelope and linearly scaled with the input across all speech rates, irrespective of intelligibility. In contrast, secondary auditory areas in the STG as well as the inferior frontal gyrus and angular gyrus ( high-level) tracked the acoustic envelope and linearly scaled with input only for intelligible speech. These results help reconcile seemingly contradictory previous findings and provide better understanding of how information processing unfolds along the cortical auditory hierarchy.

neuroscience↗

bioRxiv2018

Propagation of information along the cortical hierarchy as a function of attention while reading and listening to stories

How does attention route information from sensory to high-order areas as a function of task, within the relatively fixed topology of the brain? In this study, participants were simultaneously presented with two unrelated stories - one spoken and one written - and asked to attend one while ignoring the other. We used fMRI and a novel inter-subject correlation analysis to track the spread of information along the processing hierarchy as a function of task. Processing the unattended spoken (written) information was confined to auditory (visual) cortices. In contrast, attending to the spoken (written) story enhanced the stimulus-selective responses in early sensory regions and allowed it to spread into higher-order areas. Surprisingly, we found that the story-specific spoken (written) responses for the attended story also reached the opposite secondary visual (auditory) regions. These results demonstrate how attention enhances the processing of attended input and allows it to propagate across brain areas.

neuroscience↗

bioRxiv2018

Task-induced deactivation in diverse brain systems correlates with interindividual differences in distinct autonomic indices

Neuroimaging research has shown that different cognitive tasks induce relatively specific activation patterns, as well as less task-specific deactivation patterns. Here we examined whether individual differences in Autonomic Nervous System (ANS) activity during task performance correlate with the magnitude of task-induced deactivation. In an fMRI study, participants performed a continuous mental arithmetic task in a task/rest block design, while undergoing combined fMRI and heart / respiration rate acquisitions using photoplethysmograph and respiration belt. As expected, task performance increased heart-rate and reduced the RMSSD, a cardiac index related to vagal tone. Across participants, higher heart rate during task was linked to increased activation in fronto-parietal regions, as well as to stronger deactivation in ventromedial prefrontal regions. Respiration frequency during task was associated with similar patterns, but in different regions than those identified for heart-rate. Finally, in a large set of regions, almost exclusively limited to the Default Mode Network, lower RMSSD was associated with greater deactivation, and furthermore, the vast majority of these regions were task-deactivated at the group level. Together, our findings show that inter-individual differences in ANS activity are strongly linked to task-induced deactivation. Importantly, our findings suggest that deactivation is a multifaceted construct potentially linked to ANS control, because distinct ANS measures correlate with deactivation in different regions. We discuss the implications for current theories of cortical control of the ANS and for accounts of deactivation, with particular reference to studies documenting a \"failure to deactivate\" in multiple clinical states.

neuroscience↗

bioRxiv2018

Representation of real-world event schemas during narrative perception

Understanding movies and stories requires maintaining a high-level situation model that abstracts away from perceptual details to describe the location, characters, actions, and causal relationships of the currently unfolding event. These models are built not only from information present in the current narrative, but also from prior knowledge about schematic event scripts, which describe typical event sequences encountered throughout a lifetime. We analyzed fMRI data from 44 human subjects presented with sixteen three-minute stories, consisting of four schematic events drawn from two different scripts (eating at a restaurant or going through the airport). Aside from this shared script structure, the stories varied widely in terms of their characters and storylines, and were presented in two highly dissimilar formats (audiovisual clips or spoken narration). One group was presented with the stories in an intact temporal sequence, while a separate control group was presented with the same events in scrambled order. Regions including the posterior medial cortex, medial prefrontal cortex (mPFC), and superior frontal gyrus exhibited schematic event patterns that generalized across stories, subjects, and modalities. Patterns in mPFC were also sensitive to overall script structure, with temporally scrambled events evoking weaker schematic representations. Using a Hidden Markov Model, patterns in these regions can predict the script (restaurant vs. airport) of unlabeled data with high accuracy, and can be used to temporally align multiple stories with a shared script. These results extend work on the perception of controlled, artificial schemas in human and animal experiments to naturalistic perception of complex narrative stimuli.\n\nSignificance StatementIn almost all situations we encounter in our daily lives, we are able to draw on our schematic knowledge about what typically happens in the world to better perceive and mentally represent our ongoing experiences. In contrast to previous studies that investigated schematic cognition using simple, artificial associations, we measured brain activity from subjects watching movies and listening to stories depicting restaurant or airport experiences. Our results reveal a network of brain regions that is sensitive to the shared temporal structure of these naturalistic situations. These regions abstract away from the particular details of each story, activating a representation of the general type of situation being perceived.

neuroscience↗

bioRxiv2018

Cross-modal and non-monotonic representations of statistical regularity are encoded in local neural response patterns

Current neurobiological models assign a central role to predictive processes calibrated to environmental statistics. Neuroimaging studies examining the encoding of stimulus uncertainty have relied almost exclusively on manipulations in which stimuli were presented in a single sensory modality, and further assumed that neural responses vary monotonically with uncertainty. This has left a gap in theoretical development with respect to two core issues: i) are there cross-modal brain systems that encode input uncertainty in way that generalizes across sensory modalities, and ii) are there brain systems that track input uncertainty in a non-monotonic fashion? We used multivariate pattern analysis to address these two issues using auditory, visual and audiovisual inputs. We found signatures of cross-modal encoding in frontoparietal, orbitofrontal, and association cortices using a searchlight cross-classification analysis where classifiers trained to discriminate levels of uncertainty in one modality were tested in another modality. Additionally, we found widespread systems encoding uncertainty non-monotonically using classifiers trained to discriminate intermediate levels of uncertainty from both the highest and lowest uncertainty levels. These findings comprise the first comprehensive report of cross-modal and non-monotonic neural sensitivity to statistical regularities in the environment, and suggest that conventional paradigms testing for monotonic responses to uncertainty in a single sensory modality may have limited generalizability.

neuroscience↗

bioRxiv2017

Skilled lipreaders read and listen to lips

Only a few of us are skilled lipreaders while most struggle at the task. To illuminate the poorly understood neural substrate of this variability, we estimated the similarity of brain activity during lipreading, listening, and reading of the same 8-min narrative with subjects whose lipreading skill varied extensively. The similarity of brain activity was estimated by voxel-wise comparison of the BOLD signal time courses. Inter-subject correlation of the time courses revealed that lipreading and listening are supported by the same brain areas in temporal, parietal and frontal cortices, precuneus and cerebellum. However, lipreading activated only a small part of the neural network that is active during listening/reading the narrative, demonstrating that neural processing during lipreading vs. listening/reading differs substantially. Importantly, skilled lipreading was specifically associated with bilateral activity in the superior and middle temporal cortex, which also encode auditory speech. Our novel results both confirm previous results from few previous studies using isolated speech segments as stimuli but also extend in an important way understanding of neural mechanisms of lipreading.

neuroscience↗

bioRxiv2017

Shared understanding is correlated with shared neural responses in the default mode network

Humans have a striking ability to infer meaning from even the sparsest and most abstract forms of narratives. At the same time, flexibility in the form of a narrative is matched by inherent ambiguity in the interpretation of it. How does the brain represent subtle, idiosyncratic differences in the interpretation of abstract and ambiguous narratives? In this fMRI study, we scanned subjects watching a 7-min original animation that depicts a complex narrative through the movement of simple geometric shapes. We additionally scanned two separate groups listening to concrete verbal descriptions of either the social narrative or the physical properties of the movie. After scanning, all subjects freely recalled their interpretation of the stimulus. Using an intersubject representational similarity analysis, we compared the similarity of narrative interpretation across subjects, as measured using text analysis, with the similarity of neural responses, as measured using intersubject correlation (ISC). We found that the more similar two peoples interpretations of the abstract shape movie were, the more similar their neural responses were in the default mode network (DMN). Moreover, these shared responses were modality invariant: despite vast differences in stimulus properties, we found that the shapes movie and the verbal interpretation of the movie elicited shared responses in linguistic areas and a subset of the DMN when subjects shared interpretations. Together, these results suggest that the DMN is not only sensitive to subtle individual differences in narrative interpretation, but also resilient to large differences in the modality of the narrative.\n\nSignificance statementThe same narrative can be both communicated in different ways and interpreted in different ways. How are subtle, idiosyncratic differences in the interpretation of complex narratives presented in different forms represented in the brain? In this fMRI study, we show that the more similarly two people interpreted an ambiguous animation composed of moving shapes, the more similar their neural responses were in the Default Mode Network. In addition, by presenting the same narrative in a different form, we found shared responses across modalities when subjects shared interpretations despite the vast differences in modality of the stimuli. Together, these results suggest that the DMN is at once sensitive to individual differences in narrative interpretation and resilient to differences narrative modality.

neuroscience↗

bioRxiv2017

Elucidating relations between fMRI, ECoG and EEG through a common natural stimulus

Human brain mapping relies heavily on fMRI, ECoG and EEG, which capture different physiological signals. Relationships between these signals have been established in the context of specific tasks or during resting state, often using spatially confined concurrent recordings in animals. But it is not certain whether these correlations generalize to other contexts relevant for human cognitive neuroscience. Here, we address the case of complex naturalistic stimuli and ask two basic questions. First, how reliable are the responses evoked by a naturalistic audio-visual stimulus in each of these imaging methods, and second, how similar are stimulus-related responses across methods? To this end, we investigated a wide range of brain regions and frequency bands. We presented the same movie clip twice to three different cohorts of subjects (NEEG = 45, NfMRI = 11, NECoG = 5) and assessed stimulus-driven correlations across viewings and between imaging methods, thereby ruling out task-irrelevant confounds. All three imaging methods had similar repeat-reliability across viewings when fMRI and EEG data were averaged across subjects, highlighting the potential to achieve large signal-to-noise ratio by leveraging large sample sizes. The fMRI signal correlated positively with high-frequency ECoG power across multiple task-related cortical structures but positively with low-frequency EEG and ECoG power. In contrast to previous studies, these correlations were as strong for low-frequency as for high frequency ECoG. We also observed links between fMRI and infra-slow EEG voltage fluctuations. These results extend previous findings to the case of natural stimulus processing.

neuroscience↗

bioRxiv2017

Learning naturalistic temporal structure in the posterior medial network

The posterior medial network is at the apex of a temporal integration hierarchy in the brain, integrating information over many seconds of viewing intact, but not scrambled, movies. This has been interpreted as an effect of temporal structure. Such structure in movies depends on pre-existing event schemas, but temporal structure can also arise de novo from learning. Here we examined the relative role of schema-consistent temporal structure and arbitrary but consistent temporal structure on the human posterior medial network. We tested whether, with repeated viewing, the network becomes engaged by scrambled movies with temporal structure. Replicating prior studies, posterior medial regions were immediately locked to stimulus structure upon exposure to intact but not scrambled movies. However, for temporally structured scrambled movies, functional coupling within the network increased across stimulus repetitions, rising to the level of intact movies. Thus, temporal structure is a key determinant of network dynamics and function.

neuroscience↗

bioRxiv2017

A Core Speech Circuit Between Primary Motor, Somatosensory, And Auditory Cortex: Evidence From Connectivity And Genetic Descriptions

What adaptations allow humans to produce and perceive speech so effortlessly? We show that speech is supported by a largely undocumented core of structural and functional connectivity between the central sulcus (CS or primary motor and somatosensory cortex) and the transverse temporal gyrus (TTG or primary auditory cortex). Anatomically, we show that CS and TTG cortical thickness covary across individuals and that they are connected by white matter tracts. Neuroimaging network analyses confirm the functional relevance and specificity of these structural relationships. Specifically, the CS and TTG are functionally connected at rest, during natural audiovisual speech perception, and are coactive over a large variety of linguistic stimuli and tasks. Importantly, across structural and functional analyses, connectivity of regions immediately adjacent to the TTG are with premotor and prefrontal regions rather than the CS. Finally, we show that this structural/functional CS-TTG relationship is mediated by a constellation of genes associated with vocal learning and disorders of efference copy. We propose that this core circuit constitutes an interface for rapidly exchanging articulatory and acoustic information and discuss implications for current models of speech.

neuroscience↗

bioRxiv2017

A probabilistic approach to discovering dynamic full-brain functional connectivity patterns

Recent research shows that the covariance structure of functional magnetic resonance imaging (fMRI) data - commonly described as functional connectivity - can change as a function of the participants cognitive state (for review see [35]). Here we present a Bayesian hierarchical matrix factorization model, termed hierarchical topographic factor analysis (HTFA), for efficiently discovering full-brain networks in large multi-subject neuroimaging datasets. HTFA approximates each subjects network by first re-representing each brain image in terms of the activities of a set of localized nodes, and then computing the covariance of the activity time series of these nodes. The number of nodes, along with their locations, sizes, and activities (over time) are learned from the data. Because the number of nodes is typically substantially smaller than the number of fMRI voxels, HTFA can be orders of magnitude more efficient than traditional voxel-based functional connectivity approaches. In one case study, we show that HTFA recovers the known connectivity patterns underlying a collection of synthetic datasets. In a second case study, we illustrate how HTFA may be used to discover dynamic full-brain activity and connectivity patterns in real fMRI data, collected as participants listened to a story. In a third case study, we carried out a similar series of analyses on fMRI data collected as participants viewed an episode of a television show. In these latter case studies, we found that the HTFA-derived activity and connectivity patterns can be used to reliably decode which moments in the story or show the participants were experiencing. Further, we found that these two classes of patterns contained partially non-overlapping information, such that decoders trained on combinations of activity-based and dynamic connectivity-based features performed better than decoders trained on activity or connectivity patterns alone. We replicated this latter result with two additional (previously developed) methods for efficiently characterizing full-brain activity and connectivity patterns.

neuroscience↗

bioRxiv2017

The butterfly effect: amplification of local changes along the temporal processing hierarchy

Changing just a few words in a story can induce a substantial change in the overall narrative. How does the brain accumulate and process local and sparse changes, creating a unique situation model of the story, over the course of a real-life narrative? Recently, we mapped a hierarchy of processing timescales in the brain: from early sensory areas that integrate information over 10s-100s ms, to high-order areas that integrate information over many seconds to minutes. Based on this hierarchy, we hypothesize that early sensory areas would be sensitive to local changes in word use, but that there will be increasingly divergent neural responses along the processing hierarchy as higher-order areas accumulate and amplify these local changes. To test this hypothesis, we created two structurally related but interpretively distinct narratives by changing some individual words. We found that the neural response distance between the stories was amplified as story information is transferred from low-level regions (e.g. early auditory cortex) to high-level regions (e.g precuneus and prefrontal cortex) and that the neural difference between stories is highly correlated with an areas ability to integrate information over time. Our results suggest a neural mechanism by which two similar situations become easy to distinguish.

neuroscience↗

bioRxiv2017

Altered topology of neural circuits in congenital prosopagnosia

Using a novel fMRI-based inter-subject functional correlation (ISFC) approach, which isolates stimulus-locked inter-regional correlation patterns, we compared the cortical topology of the neural circuit for face processing in participants with congenital prosopagnosia (CP) and matched controls. Whereas the anterior temporal lobe served as the major network hub for face processing in controls, this was not the case for the CPs. Instead, this group evinced hyper-connectivity in posterior regions of the visual cortex, mostly associated with the lateral occipital and the inferior temporal cortices. Moreover, the extent to which the network organization was atypical differed as a function of the severity of the face recognition deficit. These results offer new insights into the perturbed cortical topology in CP, which may serve as the underlying neural basis of the behavioral deficits typical of this disorder. The approach adopted here has the potential to uncover altered topologies in other neurodevelopmental disorders, as well.\n\nSignificance StatementCongenital prosopagnosia (CP; face blindness), a developmental deficit in face recognition, is thought to affect up to 3% of the population. Understanding its neural basis is challenging as there is no obvious deficit on conventional structural or functional MRI scans. Using an innovative, fMRI-based inter-subject correlation approach geared towards tracking inter-regional stimulus-locked brain activation, the present study uncovers marked topological differences in a distributed brain network of higher-order visual regions in CP relative to controls. Alteration in topology also differs as a function of the severity of the deficit. These findings shed new light on the neural perturbations underlying CP, and the analytic approach we have adopted may have utility in elucidating the neural basis of other neurodevelopmental disorders such as dyslexia or amusia.

neuroscience↗

bioRxiv2016

Measuring speaker-listener neural coupling with functional near infrared spectroscopy

The present study investigates brain-to-brain coupling, defined as inter-subject correlations in the hemodynamic response, during natural verbal communication. We used functional near-infrared spectroscopy (fNIRS) to record brain activity of speakers telling stories and listeners comprehending audio recordings of these stories. Listeners brain activity was correlated with speakers with a delay. This between-brain correlation disappeared when verbal communication failed. We further compared the fNIRS and functional Magnetic Resonance Imaging (fMRI) recordings of listeners comprehending the same story and found a relationship between the fNIRS oxygenated-hemoglobin concentration changes and the fMRI BOLD in brain areas associated with speech comprehension. This correlation between fNIRS and fMRI was only present when data from the same story were compared between the two modalities and vanished when data from different stories were compared; this cross-modality consistency further highlights the reliability of the spatiotemporal brain activation pattern as a measure of story comprehension. Our findings suggest that fNIRS is a powerful tool for investigating brain-to-brain coupling during verbal communication. As fNIRS sensors are relatively low-cost and can even be built into wireless, portable, battery-operated systems, these results highlight the potential of broad utilization of this approach in everyday settings for augmenting communication and interaction.

neuroscience↗

bioRxiv2016

How we transmit memories to other brains: constructing shared neural representations via communication

It is striking that humans are able to encode and later verbally share their memories of an episode with listeners, who are in turn able to imagine (mentally construct) details of the episode which they have not personally experienced. However, it is unknown how strongly the neural patterns elicited by imagining specific episodes resemble the neural states elicited during the original encoding of those episodes. In the current study, using fMRI and a natural communication task, we traced how neural patterns associated with specific scenes depicted in a movie are encoded, verbally recalled, and then transferred to a group of naive listeners who construct the scenes of the movie in their imagination. By comparing neural patterns across the three conditions, we report, for the first time, that event-specific neural patterns are observed in the default mode network (DMN) and shared across the encoding, reinstatement (spoken recall), and new construction (imagination) of the same real-life episode. This study uncovers the intimate correspondences between memory encoding and imagination, and highlights the essential role that our common language plays in the process of transmitting ones experiences to other brains.

neuroscience↗

bioRxiv2016

Discovering event structure in continuous narrative perception and memory

SummaryDuring realistic, continuous perception, humans automatically segment experiences into discrete events. Using a novel model of neural event dynamics, we investigate how cortical structures generate event representations during continuous narratives, and how these events are stored and retrieved from long-term memory. Our data-driven approach enables identification of event boundaries and event correspondences across datasets without human-generated stimulus annotations, and reveals that different regions segment narratives at different timescales. We also provide the first direct evidence that narrative event boundaries in high-order areas (overlapping the default mode network) trigger encoding processes in the hippocampus, and that this encoding activity predicts pattern reinstatement during recall. Finally, we demonstrate that these areas represent abstract, multimodal situation models, and show anticipatory event reinstatement as subjects listen to a familiar narrative. Our results provide strong evidence that brain activity is naturally structured into semantically meaningful events, which are stored in and retrieved from long-term memory.

neuroscience↗