Biology subjects

Hasson, U.

Publications and source records attributed to Hasson, U..

At least 55 records · Page 3Linked to original sources

bioRxiv2019

Robust-fit to nature: an evolutionary perspective on biological (and artificial) neural networks

Evolution is a blind fitting process by which organisms, over generations, adapt to the niches of an ever-changing environment. Does the mammalian brain use similar brute-force fitting processes to learn how to perceive and act upon the world? Recent advances in training deep neural networks has exposed the power of optimizing millions of synaptic weights to map millions of observations along ecologically relevant objective functions. This class of models has dramatically outstripped simpler, more intuitive models, operating robustly in real-life contexts spanning perception, language, and action coordination. These models do not learn an explicit, human-interpretable representation of the underlying structure of the data; rather, they use local computations to interpolate over task-relevant manifolds in a high-dimensional parameter space. Furthermore, counterintuitively, over-parameterized models, similarly to evolutionary processes, can be simple and parsimonious as they provide a versatile, robust solution for learning a diverse set of functions. In contrast to traditional scientific models, where the ultimate goal is interpretability, over-parameterized models eschew interpretability in favor of solving real-life problems or tasks. We contend that over-parameterized blind fitting presents a radical challenge to many of the underlying assumptions and practices in computational neuroscience and cognitive psychology. At the same time, this shift in perspective informs longstanding debates and establishes unexpected links with evolution, ecological psychology, and artificial life.

neuroscience↗

bioRxiv2019

Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space

Connectivity hyperalignment can be used to estimate a single shared response space across disjoint datasets. We develop a connectivity-based shared response model that factorizes aggregated fMRI datasets into a single reduced-dimension shared connectivity space and subject-specific topographic transformations. These transformations resolve idiosyncratic functional topographies and can be used to project response time series into shared space. We evaluate this algorithm on a large collection of heterogeneous, naturalistic fMRI datasets acquired while subjects listened to spoken stories. Projecting subject data into shared space dramatically improves between-subject story time-segment classification and increases the dimensionality of shared information across subjects. This improvement generalizes to subjects and stories excluded when estimating the shared space. We demonstrate that estimating a simple semantic encoding model in shared space improves between-subject forward encoding and inverted encoding model performance. The shared space estimated across all datasets is distinct from the shared space derived from any particular constituent dataset; the algorithm leverages shared connectivity to yield a consensus shared space conjoining diverse story stimuli.\n\nHighlightsO_LIConnectivity SRM estimates a single shared space across subjects and stimuli\nC_LIO_LITopographic transformations resolve idiosyncrasies across individuals\nC_LIO_LIShared connectivity space enhances spatiotemporal intersubject correlations\nC_LIO_LISemantic model-based encoding and decoding improves across subjects\nC_LIO_LITransformations project into a consensus space conjoining diverse stimuli\nC_LI

neuroscience↗

bioRxiv2019

Temporal integration of narrative information in a hippocampal amnesic patient

Default network regions appear to integrate information over time windows of 30 seconds or more during narrative listening. Does this long-timescale capability require the hippocampus? Amnesic behavior suggests that the hippocampus may not be needed for online processing when input is continuous and semantically rich: amnesics can participate in conversations and tell stories spanning minutes, and when tested immediately on recently heard prose their performance is relatively preserved. We hypothesized that default network regions can integrate the semantically coherent information of a narrative across long time windows, even in the absence of the hippocampus. To test this prediction, we measured BOLD activity in the brain of a hippocampal amnesic patient (D. A.) and healthy control participants while they listened to a seven-minute narrative. The narrative was played either in its intact form, or as a paragraph-scrambled version, which has been previously shown to interfere with the long-range temporal dependencies in default network activity. In the intact story condition, D. A.s moment-by-moment BOLD activity spatial patterns were similar to those of controls in low-level auditory cortex as well as in some high-level default network regions (including lateral and medial posterior parietal cortex). Moreover, as in controls, D. A.s response patterns in medial and lateral posterior parietal cortex were disrupted when paragraphs of the story were presented in a shuffled order, suggesting that activity in these areas did depend on information from 30 seconds or more in the past. Together, these results suggest that some default network cortical areas can integrate information across long timescales, even in the absence of the hippocampus.

neuroscience↗

bioRxiv2019

Age-Related Changes in Neural Networks Supporting Complex Visual and Social Processing in Adolescence

Despite our differences, there is much about the natural visual world that almost all observers apparently perceive in common. This coherence across observers is evidenced by the finding that, across adults, approximately 30% of the brain is activated in a consistent fashion in response to viewing naturalistic input. The critical question addressed here is how does this consistency emerge and is this pattern of coherence apparent from early in development or does it evolve with time and/or experience? We focused our investigation at a key developmental juncture that might bridge the child and adult patterns, namely, the period of adolescence. We acquired fMRI BOLD data evoked by an 11-minute age-appropriate movie in younger (age 9-14 years) and older adolescents (age 15-19 years) and in adults. Using an intra-subject correlation approach, we characterized the consistency of the neural response within-individual (across two separate runs of the movie), and then, using an inter-subject correlation approach, evaluated the similarity of the response profile within individuals of the same age group and between age-groups. In primary sensory areas (A1+, V1) the response profiles in both groups of adolescents were highly similar to those of the adults, suggesting that these areas are functionally mature at earlier stages of the development. In contrast, some other regions exhibited higher within-age correlations in the adolescent groups than in the adult group. Last, we evaluated the brain responses across the whole cortex and identified the different patterns of maturation as reflected in different inter-subject correlations across the age groups. Together, these findings provide a fine-grained characterization of functional neural development. The approach offers the potential for careful tracking of the development of widespread cortical networks that support the emerging stereotypical responses to naturalistic visual and social stimuli and has important implications for future studies of cortical development.

neuroscience↗

bioRxiv2019

Spatial and semantic regularities produce interactive effects in early stages of visual orientation

Learning environmental regularities allows predicting multiple dimensions of future events such as their location and semantic features. However, few studies have examined how multi-dimensional predictions are implemented, and mechanistic accounts are absent. Using eye tracking study, we evaluated whether predictions of object-location and object-category interact during the earliest stages of orientation. We presented stochastic series so that across four conditions, participants could predict either the location of the next image, its semantic category, both dimensions, or neither. Participants observed images in absence of any task. We modeled saccade latencies using ELATER, a rise-to-threshold model that accounts for accumulation rate (AR), variance of AR over trials, and decision threshold. The main findings were: 1) accumulation-rate scaled with the degree of surprise associated with location of target-presentation (confirmatory result); 2) predictability of semantic-category hindered latencies, but only when images were presented at a surprising location, suggesting a bottleneck in implementing joint predictions; 3) saccades to images that satisfied semantic expectations were associated with larger variance of accumulation-rate than saccades to semantically-surprising images, consistent with a richer repertoire of early evaluative processes for semantically-expected images. Joint impacts of location and target-identity regularity were also identified in analyses of anticipatory fixation offsets. The results indicate a strong interaction between the processing of regularities in object location and identity during stimulus-guided saccades, and suggest these regularities also impact anticipatory, non-stimulus-guided processes.

neuroscience↗

bioRxiv2019

Measuring shared responses across subjects using intersubject correlation

Our capacity to jointly represent information about the world underpins our social experience. By leveraging one individuals brain activity to model anothers, we can measure shared information across brains--even in dynamic, naturalistic scenarios where an explicit response model may be unobtainable. Introducing experimental manipulations allows us to measure, for example, shared responses between speakers and listeners, or between perception and recall. In this tutorial, we develop the logic of intersubject correlation (ISC) analysis and discuss the family of neuroscientific questions that stem from this approach. We also extend this logic to spatially distributed response patterns and functional network estimation. We provide a thorough and accessible treatment of methodological considerations specific to ISC analysis, and outline best practices.

neuroscience↗

bioRxiv2018

Infant and adult brains are coupled to the dynamics of natural communication

Infancy is the foundational period for learning from adults, and the dynamics of the social environment have long been proposed as central to childrens development. Here we reveal a novel, highly naturalistic approach for studying live interactions between infants and adults. Using functional near-infrared spectroscopy (fNIRS), we simultaneously and continuously measured the brains of infants (9-15 months) and an adult while they communicated and played with each other in real time. We found that time-locked neural coupling within dyads was significantly greater when they interacted with each other than with control individuals. In addition, we found that both infant and adult brains continuously tracked the moment-to-moment fluctuations of mutual gaze, infant emotion, and adult speech prosody with high temporal precision. This investigation advances what is currently known about how the brains and behaviors of infants both shape and reflect those of adults during real-life communication.

neuroscience↗

bioRxiv2018

Electrocorticographic responses to time-compressed speech vary across the cortical auditory hierarchy

Human listeners understand spoken language across a variety of rates, but when speech is presented three times or more faster than its usual rate, it becomes unintelligible. How the brain achieves such tolerance and why speech becomes unintelligible above certain rates is still unclear. We addressed these questions using electrocorticography (ECoG) recordings in 7 epileptic patients (two female). Patients rated the intelligibility of sentences presented at the original rate (100%), speeded rates (33% or 66% of the original sentence duration) and a slowed rate (150%). We then examined which parameters of the neural response covary with the transition from intelligible to unintelligible speech. Specifically, we asked whether neural responses: 1) track the acoustic envelope of the incoming speech; 2) \"scale\" with speech rate, i.e. whether neural responses elicited by slowed and speeded sentences can be linearly scaled to match the responses to the original sentence. Behaviorally, intelligibility was at ceiling for speech rates of 66% and above, but dropped significantly for the 33% rate. At the neural level, Superior Temporal Gyrus regions (STG) in close proximity to A1 ( low-level) tracked the acoustic envelope and linearly scaled with the input across all speech rates, irrespective of intelligibility. In contrast, secondary auditory areas in the STG as well as the inferior frontal gyrus and angular gyrus ( high-level) tracked the acoustic envelope and linearly scaled with input only for intelligible speech. These results help reconcile seemingly contradictory previous findings and provide better understanding of how information processing unfolds along the cortical auditory hierarchy.

neuroscience↗

bioRxiv2018

Propagation of information along the cortical hierarchy as a function of attention while reading and listening to stories

How does attention route information from sensory to high-order areas as a function of task, within the relatively fixed topology of the brain? In this study, participants were simultaneously presented with two unrelated stories - one spoken and one written - and asked to attend one while ignoring the other. We used fMRI and a novel inter-subject correlation analysis to track the spread of information along the processing hierarchy as a function of task. Processing the unattended spoken (written) information was confined to auditory (visual) cortices. In contrast, attending to the spoken (written) story enhanced the stimulus-selective responses in early sensory regions and allowed it to spread into higher-order areas. Surprisingly, we found that the story-specific spoken (written) responses for the attended story also reached the opposite secondary visual (auditory) regions. These results demonstrate how attention enhances the processing of attended input and allows it to propagate across brain areas.

neuroscience↗

bioRxiv2018

Task-induced deactivation in diverse brain systems correlates with interindividual differences in distinct autonomic indices

Neuroimaging research has shown that different cognitive tasks induce relatively specific activation patterns, as well as less task-specific deactivation patterns. Here we examined whether individual differences in Autonomic Nervous System (ANS) activity during task performance correlate with the magnitude of task-induced deactivation. In an fMRI study, participants performed a continuous mental arithmetic task in a task/rest block design, while undergoing combined fMRI and heart / respiration rate acquisitions using photoplethysmograph and respiration belt. As expected, task performance increased heart-rate and reduced the RMSSD, a cardiac index related to vagal tone. Across participants, higher heart rate during task was linked to increased activation in fronto-parietal regions, as well as to stronger deactivation in ventromedial prefrontal regions. Respiration frequency during task was associated with similar patterns, but in different regions than those identified for heart-rate. Finally, in a large set of regions, almost exclusively limited to the Default Mode Network, lower RMSSD was associated with greater deactivation, and furthermore, the vast majority of these regions were task-deactivated at the group level. Together, our findings show that inter-individual differences in ANS activity are strongly linked to task-induced deactivation. Importantly, our findings suggest that deactivation is a multifaceted construct potentially linked to ANS control, because distinct ANS measures correlate with deactivation in different regions. We discuss the implications for current theories of cortical control of the ANS and for accounts of deactivation, with particular reference to studies documenting a \"failure to deactivate\" in multiple clinical states.

neuroscience↗

bioRxiv2018

Representation of real-world event schemas during narrative perception

Understanding movies and stories requires maintaining a high-level situation model that abstracts away from perceptual details to describe the location, characters, actions, and causal relationships of the currently unfolding event. These models are built not only from information present in the current narrative, but also from prior knowledge about schematic event scripts, which describe typical event sequences encountered throughout a lifetime. We analyzed fMRI data from 44 human subjects presented with sixteen three-minute stories, consisting of four schematic events drawn from two different scripts (eating at a restaurant or going through the airport). Aside from this shared script structure, the stories varied widely in terms of their characters and storylines, and were presented in two highly dissimilar formats (audiovisual clips or spoken narration). One group was presented with the stories in an intact temporal sequence, while a separate control group was presented with the same events in scrambled order. Regions including the posterior medial cortex, medial prefrontal cortex (mPFC), and superior frontal gyrus exhibited schematic event patterns that generalized across stories, subjects, and modalities. Patterns in mPFC were also sensitive to overall script structure, with temporally scrambled events evoking weaker schematic representations. Using a Hidden Markov Model, patterns in these regions can predict the script (restaurant vs. airport) of unlabeled data with high accuracy, and can be used to temporally align multiple stories with a shared script. These results extend work on the perception of controlled, artificial schemas in human and animal experiments to naturalistic perception of complex narrative stimuli.\n\nSignificance StatementIn almost all situations we encounter in our daily lives, we are able to draw on our schematic knowledge about what typically happens in the world to better perceive and mentally represent our ongoing experiences. In contrast to previous studies that investigated schematic cognition using simple, artificial associations, we measured brain activity from subjects watching movies and listening to stories depicting restaurant or airport experiences. Our results reveal a network of brain regions that is sensitive to the shared temporal structure of these naturalistic situations. These regions abstract away from the particular details of each story, activating a representation of the general type of situation being perceived.

neuroscience↗

bioRxiv2018

Cross-modal and non-monotonic representations of statistical regularity are encoded in local neural response patterns

Current neurobiological models assign a central role to predictive processes calibrated to environmental statistics. Neuroimaging studies examining the encoding of stimulus uncertainty have relied almost exclusively on manipulations in which stimuli were presented in a single sensory modality, and further assumed that neural responses vary monotonically with uncertainty. This has left a gap in theoretical development with respect to two core issues: i) are there cross-modal brain systems that encode input uncertainty in way that generalizes across sensory modalities, and ii) are there brain systems that track input uncertainty in a non-monotonic fashion? We used multivariate pattern analysis to address these two issues using auditory, visual and audiovisual inputs. We found signatures of cross-modal encoding in frontoparietal, orbitofrontal, and association cortices using a searchlight cross-classification analysis where classifiers trained to discriminate levels of uncertainty in one modality were tested in another modality. Additionally, we found widespread systems encoding uncertainty non-monotonically using classifiers trained to discriminate intermediate levels of uncertainty from both the highest and lowest uncertainty levels. These findings comprise the first comprehensive report of cross-modal and non-monotonic neural sensitivity to statistical regularities in the environment, and suggest that conventional paradigms testing for monotonic responses to uncertainty in a single sensory modality may have limited generalizability.

neuroscience↗

bioRxiv2017

Skilled lipreaders read and listen to lips

Only a few of us are skilled lipreaders while most struggle at the task. To illuminate the poorly understood neural substrate of this variability, we estimated the similarity of brain activity during lipreading, listening, and reading of the same 8-min narrative with subjects whose lipreading skill varied extensively. The similarity of brain activity was estimated by voxel-wise comparison of the BOLD signal time courses. Inter-subject correlation of the time courses revealed that lipreading and listening are supported by the same brain areas in temporal, parietal and frontal cortices, precuneus and cerebellum. However, lipreading activated only a small part of the neural network that is active during listening/reading the narrative, demonstrating that neural processing during lipreading vs. listening/reading differs substantially. Importantly, skilled lipreading was specifically associated with bilateral activity in the superior and middle temporal cortex, which also encode auditory speech. Our novel results both confirm previous results from few previous studies using isolated speech segments as stimuli but also extend in an important way understanding of neural mechanisms of lipreading.

neuroscience↗

bioRxiv2017

Shared understanding is correlated with shared neural responses in the default mode network

Humans have a striking ability to infer meaning from even the sparsest and most abstract forms of narratives. At the same time, flexibility in the form of a narrative is matched by inherent ambiguity in the interpretation of it. How does the brain represent subtle, idiosyncratic differences in the interpretation of abstract and ambiguous narratives? In this fMRI study, we scanned subjects watching a 7-min original animation that depicts a complex narrative through the movement of simple geometric shapes. We additionally scanned two separate groups listening to concrete verbal descriptions of either the social narrative or the physical properties of the movie. After scanning, all subjects freely recalled their interpretation of the stimulus. Using an intersubject representational similarity analysis, we compared the similarity of narrative interpretation across subjects, as measured using text analysis, with the similarity of neural responses, as measured using intersubject correlation (ISC). We found that the more similar two peoples interpretations of the abstract shape movie were, the more similar their neural responses were in the default mode network (DMN). Moreover, these shared responses were modality invariant: despite vast differences in stimulus properties, we found that the shapes movie and the verbal interpretation of the movie elicited shared responses in linguistic areas and a subset of the DMN when subjects shared interpretations. Together, these results suggest that the DMN is not only sensitive to subtle individual differences in narrative interpretation, but also resilient to large differences in the modality of the narrative.\n\nSignificance statementThe same narrative can be both communicated in different ways and interpreted in different ways. How are subtle, idiosyncratic differences in the interpretation of complex narratives presented in different forms represented in the brain? In this fMRI study, we show that the more similarly two people interpreted an ambiguous animation composed of moving shapes, the more similar their neural responses were in the Default Mode Network. In addition, by presenting the same narrative in a different form, we found shared responses across modalities when subjects shared interpretations despite the vast differences in modality of the stimuli. Together, these results suggest that the DMN is at once sensitive to individual differences in narrative interpretation and resilient to differences narrative modality.

neuroscience↗

bioRxiv2017

Elucidating relations between fMRI, ECoG and EEG through a common natural stimulus

Human brain mapping relies heavily on fMRI, ECoG and EEG, which capture different physiological signals. Relationships between these signals have been established in the context of specific tasks or during resting state, often using spatially confined concurrent recordings in animals. But it is not certain whether these correlations generalize to other contexts relevant for human cognitive neuroscience. Here, we address the case of complex naturalistic stimuli and ask two basic questions. First, how reliable are the responses evoked by a naturalistic audio-visual stimulus in each of these imaging methods, and second, how similar are stimulus-related responses across methods? To this end, we investigated a wide range of brain regions and frequency bands. We presented the same movie clip twice to three different cohorts of subjects (NEEG = 45, NfMRI = 11, NECoG = 5) and assessed stimulus-driven correlations across viewings and between imaging methods, thereby ruling out task-irrelevant confounds. All three imaging methods had similar repeat-reliability across viewings when fMRI and EEG data were averaged across subjects, highlighting the potential to achieve large signal-to-noise ratio by leveraging large sample sizes. The fMRI signal correlated positively with high-frequency ECoG power across multiple task-related cortical structures but positively with low-frequency EEG and ECoG power. In contrast to previous studies, these correlations were as strong for low-frequency as for high frequency ECoG. We also observed links between fMRI and infra-slow EEG voltage fluctuations. These results extend previous findings to the case of natural stimulus processing.

neuroscience↗

bioRxiv2017

Learning naturalistic temporal structure in the posterior medial network

The posterior medial network is at the apex of a temporal integration hierarchy in the brain, integrating information over many seconds of viewing intact, but not scrambled, movies. This has been interpreted as an effect of temporal structure. Such structure in movies depends on pre-existing event schemas, but temporal structure can also arise de novo from learning. Here we examined the relative role of schema-consistent temporal structure and arbitrary but consistent temporal structure on the human posterior medial network. We tested whether, with repeated viewing, the network becomes engaged by scrambled movies with temporal structure. Replicating prior studies, posterior medial regions were immediately locked to stimulus structure upon exposure to intact but not scrambled movies. However, for temporally structured scrambled movies, functional coupling within the network increased across stimulus repetitions, rising to the level of intact movies. Thus, temporal structure is a key determinant of network dynamics and function.

neuroscience↗

bioRxiv2017

A Core Speech Circuit Between Primary Motor, Somatosensory, And Auditory Cortex: Evidence From Connectivity And Genetic Descriptions

What adaptations allow humans to produce and perceive speech so effortlessly? We show that speech is supported by a largely undocumented core of structural and functional connectivity between the central sulcus (CS or primary motor and somatosensory cortex) and the transverse temporal gyrus (TTG or primary auditory cortex). Anatomically, we show that CS and TTG cortical thickness covary across individuals and that they are connected by white matter tracts. Neuroimaging network analyses confirm the functional relevance and specificity of these structural relationships. Specifically, the CS and TTG are functionally connected at rest, during natural audiovisual speech perception, and are coactive over a large variety of linguistic stimuli and tasks. Importantly, across structural and functional analyses, connectivity of regions immediately adjacent to the TTG are with premotor and prefrontal regions rather than the CS. Finally, we show that this structural/functional CS-TTG relationship is mediated by a constellation of genes associated with vocal learning and disorders of efference copy. We propose that this core circuit constitutes an interface for rapidly exchanging articulatory and acoustic information and discuss implications for current models of speech.

neuroscience↗

bioRxiv2017

A probabilistic approach to discovering dynamic full-brain functional connectivity patterns

Recent research shows that the covariance structure of functional magnetic resonance imaging (fMRI) data - commonly described as functional connectivity - can change as a function of the participants cognitive state (for review see [35]). Here we present a Bayesian hierarchical matrix factorization model, termed hierarchical topographic factor analysis (HTFA), for efficiently discovering full-brain networks in large multi-subject neuroimaging datasets. HTFA approximates each subjects network by first re-representing each brain image in terms of the activities of a set of localized nodes, and then computing the covariance of the activity time series of these nodes. The number of nodes, along with their locations, sizes, and activities (over time) are learned from the data. Because the number of nodes is typically substantially smaller than the number of fMRI voxels, HTFA can be orders of magnitude more efficient than traditional voxel-based functional connectivity approaches. In one case study, we show that HTFA recovers the known connectivity patterns underlying a collection of synthetic datasets. In a second case study, we illustrate how HTFA may be used to discover dynamic full-brain activity and connectivity patterns in real fMRI data, collected as participants listened to a story. In a third case study, we carried out a similar series of analyses on fMRI data collected as participants viewed an episode of a television show. In these latter case studies, we found that the HTFA-derived activity and connectivity patterns can be used to reliably decode which moments in the story or show the participants were experiencing. Further, we found that these two classes of patterns contained partially non-overlapping information, such that decoders trained on combinations of activity-based and dynamic connectivity-based features performed better than decoders trained on activity or connectivity patterns alone. We replicated this latter result with two additional (previously developed) methods for efficiently characterizing full-brain activity and connectivity patterns.

neuroscience↗