Biology subjects

Gazula, H.

Publications and source records attributed to Gazula, H..

16 recordsLinked to original sources

bioRxiv2024

Information-making processes in the speaker's brain drive human conversations forward

Natural human conversation is driven by the exchange of information-rich messages that surprise the listener and deviate from predictable context. While extensive research has characterized how the brain processes unexpected linguistic input during comprehension, the neural mechanisms underlying the generation of such information by the speaker remain poorly understood. In this study, we hypothesize that the speakers brain actively creates information through extra neural computation, transforming internal thoughts into novel, meaningful linguistic outputs. Utilizing a unique 24/7 dataset of continuous electrocorticography (ECoG) comprising approximately 100 hours of spontaneous natural conversations, we contrast the neural basis of speech production and comprehension within the same participants. Behaviorally, we find that speakers take longer pausing for an additional 100-150 milliseconds before producing information-rich, improbable words, even when controlling for word identity and frequency. Neurally, we provide converging evidence for a previously unreported process: generating information-rich words elicits significantly stronger neural activity (ERP) and enhanced neural encoding and decoding in language-related areas starting 100-500 ms before articulation. This pattern contrasts sharply with speech comprehension, where enhanced neural activity for predictable words occurs before onset, while responses to improbable words emerge only after onset as prediction errors. Furthermore, we demonstrate that large language models (LLMs) mirror this biological process, requiring deeper internal computation across layers to generate improbable versus probable words. Together, these results reveal that the speakers brain is not merely a transmission channel but an active generator of information, recruiting extra neural resources to create novel content that diverges from listener expectations.

neuroscience↗

bioRxiv2024

Scale matters: Large language models with billions (rather than millions) of parameters better match neural representations of natural language

Recent research has used large language models (LLMs) to study the neural basis of naturalistic language processing in the human brain. LLMs have rapidly grown in complexity, leading to improved language processing capabilities. However, neuroscience researchers havent kept up with the quick progress in LLM development. Here, we utilized several families of transformer-based LLMs to investigate the relationship between model size and their ability to capture linguistic information in the human brain. Crucially, a subset of LLMs were trained on a fixed training set, enabling us to dissociate model size from architecture and training set size. We used electrocorticography (ECoG) to measure neural activity in epilepsy patients while they listened to a 30-minute naturalistic audio story. We fit electrode-wise encoding models using contextual embeddings extracted from each hidden layer of the LLMs to predict word-level neural signals. In line with prior work, we found that larger LLMs better capture the structure of natural language and better predict neural activity. We also found a logarithmic relationship where the encoding performance peaks in relatively earlier layers as model size increases. We also observed variations in the best-performing layer across different brain regions, corresponding to an organized language processing hierarchy.

neuroscience↗

bioRxiv2023

Three-dimensional histology reveals dissociable human hippocampal long axis gradients of Alzheimer's pathology

INTRODUCTIONThree-dimensional (3D) histology analyses are essential to overcome sampling variability and understand pathological differences beyond the dissection axis. We present Path2MR, the first pipeline allowing 3D reconstruction of sparse human histology without an MRI reference. We implemented Path2MR with post-mortem hippocampal sections to explore pathology gradients in Alzheimers Disease. METHODSBlockface photographs of brain hemisphere slices are used for 3D reconstruction, from which an MRI-like image is generated using machine learning. Histology sections are aligned to the reconstructed hemisphere and subsequently to an atlas in standard space. RESULTSPath2MR successfully registered histological sections to their anatomical position along the hippocampal longitudinal axis. Combined with histopathology quantification, we found an expected peak of tau pathology at the anterior end of the hippocampus, while amyloid-{beta} displayed a quadratic anterior-posterior distribution. CONCLUSIONPath2MR, which enables 3D histology using any brain bank dataset, revealed significant differences along the hippocampus between tau and amyloid-{beta}.

neuroscience↗

bioRxiv2023

Deep speech-to-text models capture the neural basis of spontaneous speech in everyday conversations

Humans effortlessly use the continuous acoustics of speech to communicate rich linguistic meaning during everyday conversations. In this study, we leverage 100 hours (half a million words) of spontaneous open-ended conversations and concurrent high-quality neural activity recorded using electrocorticography (ECoG) to decipher the neural basis of real-world speech production and comprehension. Employing a deep multimodal speech-to-text model named Whisper, we develop encoding models capable of accurately predicting neural responses to both acoustic and semantic aspects of speech. Our encoding models achieved high accuracy in predicting neural responses in hundreds of thousands of words across many hours of left-out recordings. We uncover a distributed cortical hierarchy for speech and language processing, with sensory and motor regions encoding acoustic features of speech and higher-level language areas encoding syntactic and semantic information. Many electrodes--including those in both perceptual and motor areas--display mixed selectivity for both speech and linguistic features. Notably, our encoding model reveals a temporal progression from language-to-speech encoding before word onset during speech production and from speech-to-language encoding following word articulation during speech comprehension. This study offers a comprehensive account of the unfolding neural responses during fully natural, unbounded daily conversations. By leveraging a multimodal deep speech recognition model, we highlight the power of deep learning for unraveling the neural mechanisms of language processing in real-world contexts.

neuroscience↗

bioRxiv2023

3D analysis of dissection photographs with surface scanning and machine learning for quantitative neuropathology

We present open-source tools for 3D analysis of photographs of dissected slices of human brains, which are routinely acquired in brain banks but seldom used for quantitative analysis. Our tools can: (i) 3D reconstruct a volume from the photographs and, optionally, a surface scan; and (ii) produce a high-resolution 3D segmentation into 11 brain regions per hemisphere (22 in total), independently of the slice thickness. Our tools can be used as a substitute for ex vivo magnetic resonance imaging (MRI), which requires access to an MRI scanner, ex vivo scanning expertise, and considerable financial resources. We tested our tools on synthetic and real data from two NIH Alzheimers Disease Research Centers. The results show that our methodology yields accurate 3D reconstructions, segmentations, and volumetric measurements that are highly correlated to those from MRI. Our method also detects expected differences between post mortem confirmed Alzheimers disease cases and controls. The tools are available in our widespread neuroimaging suite "FreeSurfer" (https://surfer.nmr.mgh.harvard.edu/fswiki/PhotoTools).

neuroscience↗

bioRxiv2023

Decentralized Mixed Effects Modeling in COINSTAC

Performing group analysis on magnetic resonance imaging (MRI) data with linear mixed-effects (LME) models is challenging due to its large dimensionality and inherent multi-level covariance structure. In addition, as large-scale collaborative projects become commonplace in neuroimaging, data must increasingly be stored and analysed at different locations. In such settings, substantial overheads occur in terms of data transfer and coordination between participating research groups. In some cases, data cannot be pooled together due to privacy or regulatory concerns. In this work, we propose a decentralized LME model to perform a large-scale analysis of data from different collaborations without sharing or pooling. This method is efficient as it overcomes the hurdles of data privacy for sharing and has lower bandwidth and memory requirements for analysis than the centralized modeling approach. We evaluate our model using features extracted from structural magnetic resonance imaging (sMRI) data. Results highlight gray matter reductions in the temporal lobe/insula and medical front regions demonstrate the correctness of decentralized LME models. Our analysis also demonstrates that decentralized LME models achieve similar performance compared to the models trained with all the data in one location. We also implement the decentralized LME approach in COINSTAC, a decentralized platform for federating neuroimaging analysis, to demonstrate its value to the neuroimaging community.

neuroscience↗

bioRxiv2023

Enhancing Collaborative Neuroimaging Research: Introducing COINSTAC Vaults for Federated Analysis and Reproducibility

Collaborative neuroimaging research is often hindered by technological, policy, administrative, and methodological barriers, despite the abundance of available data. COINSTAC is a platform that successfully tackles these challenges through federated analysis, allowing researchers to analyze datasets without publicly sharing their data. This paper presents a significant enhancement to the COINSTAC platform: COINSTAC Vaults (CVs). CVs are designed to further reduce barriers by hosting standardized, persistent, and highly-available datasets, while seamlessly integrating with COINSTACs federated analysis capabilities. CVs offer a user-friendly interface for self-service analysis, streamlining collaboration and eliminating the need for manual coordination with data owners. Importantly, CVs can also be used in conjunction with open data as well, by simply creating a CV hosting the open data one would like to include in the analysis, thus filling an important gap in the data sharing ecosystem. We demonstrate the impact of CVs through several functional and structural neuroimaging studies utilizing federated analysis showcasing their potential to improve the reproducibility of research and increase sample sizes in neuroimaging studies.

neuroscience↗

bioRxiv2022

Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain

Deep language models (DLMs) provide a novel computational paradigm for how the brain processes natural language. Unlike symbolic, rule-based models described in psycholinguistics, DLMs encode words and their context as continuous numerical vectors. These "embeddings" are constructed by a sequence of computations organized in "layers" to ultimately capture surprisingly sophisticated representations of linguistic structures. How does this layered hierarchy map onto the human brain during natural language comprehension? In this study, we used electrocorticography (ECoG) to record neural activity in language areas along the superior temporal gyrus and inferior frontal gyrus while human participants listened to a 30-minute spoken narrative. We supplied this same narrative to a high-performing DLM (GPT2-XL) and extracted the contextual embeddings for each word in the story across all 48 layers of the model. We next trained a set of linear encoding models to predict the temporally-evolving neural activity from the embeddings at each layer. We found a striking correspondence between the layer-by-layer sequence of embeddings from GPT2-XL and the temporal sequence of neural activity in language areas. In addition, we found evidence for the gradual accumulation of recurrent information along the linguistic processing hierarchy. However, we also noticed additional neural processes in the brain, but not in DLMs, during the processing of surprising (unpredictable) words. These findings point to a connection between human language processing and DLMs where the layer-by-layer accumulation of contextual information in DLM embeddings matches the temporal dynamics of neural activity in high-order language areas.

neuroscience↗

bioRxiv2022

Fine temporal brain network structure modularizes and localizes differently in men and women: Insights from a novel explainability framework

AO_SCPLOWBSTRACTC_SCPLOWDeep learning has become an effective tool for classifying biological sex based on functional magnetic resonance imaging (fMRI), but research on what features within the brain are most relevant to this classification is still lacking. Model interpretability has become a powerful way to understand "black box" deep-learning models and select features within the input data that are most relevant to the correct classification. However, very little work has been done employing these methods to understand the relationship between the temporal dimension of functional imaging signals and classification of biological sex, nor has there been attention paid to rectifying problems and limitations associated with feature explanation models, e.g. underspecification and instability. We provide a methodology to limit the impact of underspecification on the stability of the measured feature importance, and then, using intrinsic connectivity networks (ICNs) from fMRI data, we provide a deep exploration of sex differences among functional brain networks. We report numerous conclusions, including activity differences in the visual and cognitive domains, as well as major connectivity differences.

neuroscience↗

bioRxiv2022

Brain embeddings with shared geometry to artificial contextual embeddings, as a code for representing language in the human brain

Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. Do language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language? To test this hypothesis, we densely recorded the neural activity in the Inferior Frontal Gyrus (IFG, also known as Brocas area) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derived for each patient a continuous vectorial representation for each word (i.e., a brain embedding). Using stringent, zero-shot mapping, we demonstrated that brain embeddings in the IFG and the DLM contextual embedding space have strikingly similar geometry. This shared geometry allows us to precisely triangulate the position of unseen words in both the brain embedding space (zero-shot encoding) and the DLM contextual embedding space (zero-shot decoding). The continuous brain embedding space provides an alternative computational framework for how natural language is represented in cortical language areas.

neuroscience↗

bioRxiv2022

Federated analysis in COINSTAC reveals functional network connectivity and spectral links to smoking and alcohol consumption in nearly 2,000 adolescent brains

With the growth of decentralized/federated analysis approaches in neuroimaging, the opportunities to study brain disorders using data from multiple sites has grown multi-fold. One such initiative is the Neuromark, a fully automated spatially constrained independent component analysis (ICA) that is used to link brain network abnormalities among different datasets, studies, and disorders while leveraging subject-specific networks. In this study, we implement the neuromark pipeline in COINSTAC, an open-source neuroimaging framework for collaborative/decentralized analysis. Decentralized analysis of nearly 2000 resting-state functional magnetic resonance imaging datasets collected at different sites across two cohorts and co-located in different countries was performed to study the resting brain functional network connectivity changes in adolescents who smoke and consume alcohol. Results showed hypoconnectivity across the majority of networks including sensory, default mode, and subcortical domains, more for alcohol than smoking, and decreased low frequency power. These findings suggest that global reduced synchronization is associated with both tobacco and alcohol use. This work demonstrates the utility and incentives associated with large-scale decentralized collaborations spanning multiple sites.

bioinformatics↗

bioRxiv2021

Decentralized Brain Age Estimation using MRI Data

Recent studies have demonstrated that neuroimaging data can be used to predict brain age, as it captures information about the neuroanatomical and functional changes the brain undergoes during development and the aging process. However, researchers often have limited access to neuroimaging data because of its challenging and expensive acquisition process, thereby limiting the effectiveness of the predictive model. Decentralized models provide a way to build more accurate and generalizable prediction models, bypassing the traditional data-sharing methodology. In this work, we propose a decentralized method for brain age estimation and evaluate it on three different feature sets, including both volumetric and voxelwise structural MRI data as well as resting functional MRI data. The results demonstrate that a decentralized brain age model can achieve similar performance compared to the models trained with all the data in one location.

neuroscience↗

bioRxiv2020

Thinking ahead: prediction in context as a keystone of language in humans and machines

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models are trained to generate appropriate linguistic responses in a given context. We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process natural language: 1) both are engaged in continuous next-word prediction before word-onset; 2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise (i.e., prediction error signals); 3) both represent words as a function of the previous context. In support of these three principles, our findings indicate that: a) the neural activity before word-onset contains context-dependent predictive information about forthcoming words, even hundreds of milliseconds before the words are perceived; b) the neural activity after word-onset reflects the surprise level and prediction error; and c) autoregressive DLM contextual embeddings capture the neural representation of context-specific word meaning better than arbitrary or static semantic embeddings. Together, our findings suggest that autoregressive DLMs provide a novel and biologically feasible computational framework for studying the neural basis of language.

neuroscience↗

bioRxiv2020

Modular and state-relevant connectivity in high-frequency resting-state BOLD fMRI data: An independent component analysis

Resting-state fMRI (rs-fMRI) data are typically filtered at different frequency bins between 0.008[~]0.2 Hz (varies across the literature) prior to analysis to mitigate nuisance variables (e.g., drift, motion, cardiac, and respiratory) and maximize the sensitivity to neuronal-mediated BOLD signal. However, multiple lines of evidence suggest meaningful BOLD signal may also be parsed at higher frequencies. To test this notion, a functional network connectivity (FNC) analysis based on a spatially informed independent component analysis (ICA) was performed at seven different bandpass frequency bins to examine FNC matrices across spectra. Further, eyes open (EO) vs. eyes closed (EC) resting-state acquisitions from the same participants were compared across frequency bins to examine if EO vs. EC FNC matrices and randomness estimations of FNC matrices are distinguishable at different frequencies. Results show that FNCs in higher-frequency bins display modular FNC similar to the lowest frequency bin, while r-to-z FNC and FNC-based measures indicating matrix non-randomness were highest in the 0.31-0.46 Hz range relative to all frequency bins above and below this range. As such, the FNC within this range appears to be the most temporally correlated, but the mechanisms facilitating this coherence require further analyses. Compared to EO, EC displayed greater FNC (involved in visual, cognitive control, somatomotor, and auditory domains) and randomness values at lower frequency bins, but this phenomenon flipped (EO > EC) at frequency bins greater than 0.46 Hz, particularly within visual regions. While the effect sizes range from small to large specific to frequency range and resting state (EO vs. EC), with little influence from common artifacts. These differences indicate that unique information can be derived from FNC between BOLD signals at different frequencies relative to a given restingstate acquisition and support the hypothesis meaningful BOLD signal is present at higher frequency ranges.

neuroscience↗

bioRxiv2019

Decentralized multi-site VBM analysis during adolescence shows structural changes linked to age, body mass index, and smoking: A COINSTAC analysis

In the recent past, there has been an upward trend in developing frameworks that enable neuroimaging researchers to address challenging questions by leveraging data across multiple sites all over the world. One such framework, Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC), provides a platform to analyze neuroimaging data stored locally across multiple organizations without the need for pooling the data at any point during the analysis. In this paper, we perform a decentralized voxel-based morphometry analysis of structural magnetic resonance imaging data across two different sites to understand the structural changes in the brain as linked to age, body mass index and smoking. Results produced by the decentralized analysis are contrasted with similar findings in literature. This work showcases the potential benefits of performing multi-voxel and multivariate analyses of large-scale neuroimaging data located at multiple sites.

neuroscience↗

bioRxiv2019

Decentralized Distribution-sampled Classification Models with Application to Brain Imaging

0.1 backgroundIn this age of big data, large data stores allow researchers to compose robust models that are accurate and informative. In many cases, the data are stored in separate locations requiring data transfer between local sites, which can cause various practical hurdles, such as privacy concerns or heavy network load. This is especially true for medical imaging data, which can be constrained due to the health insurance portability and accountability act (HIPAA). Medical imaging datasets can also contain many thousands or millions of features, requiring heavy network load.\n\n0.2 New MethodOur research expands upon current decentralized classification research by implementing a new singleshot method for both neural networks and support vector machines. Our approach is to estimate the statistical distribution of the data at each local site and pass this information to the other local sites where each site resamples from the individual distributions and trains a model on both locally available data and the resampled data.\n\n0.3 ResultsWe show applications of our approach to handwritten digit classification as well as to multi-subject classification of brain imaging data collected from patients with schizophrenia and healthy controls. Overall, the results showed comparable classification accuracy to the centralized model with lower network load than multishot methods.\n\n0.4 Comparison with Existing MethodsMany decentralized classifiers are multishot, requiring heavy network traffic. Our model attempts to alleviate this load while preserving prediction accuracy.\n\n0.5 ConclusionsWe show that our proposed approach performs comparably to a centralized approach while minimizing network traffic compared to multishot methods.\n\n0.6 HighlightsO_LIA novel yet simple approach to decentralized classification\nC_LIO_LIReduces total network load compared to current multishot algorithms\nC_LIO_LIMaintains a prediction accuracy comparable to the centralized approach\nC_LI

bioinformatics↗