Biology subjects

Zhang, R.-Y.

Publications and source records attributed to Zhang, R.-Y..

8 recordsLinked to original sources

bioRxiv2025

Predictive vision-language integration in the human visual cortex

Integrating linguistic and visual information is a core function of human cognition, yet how information from these two modalities interacts in the brain remains largely unknown. Competing frameworks, including the hub-and-spoke model and Bayesian theories such as predictive coding, offer conflicting accounts of how the brain achieves multimodal integration. To address this question, we collected a large-scale fMRI dataset and leveraged state-of-the-art AI systems to construct encoding models that probe how the human brain matches and integrates linguistic and visual information. We found that prior information from one modality can modulate neural responses in another, even in the early visual cortex (EVC). Integration neural response in EVC is governed by prediction errors consistent with predictive coding theory. Enhanced and suppressed neural responses to semantically matched cross-modal stimuli were found in distinct EVC populations, with suppression population carrying denser, behaviorally relevant semantic information. Both populations support semantic integration with distinct temporal dynamics and representational structures. These findings provide representational- and computational-level insights into how the brain integrates information across modalities, revealing unified principles of information processing that link biological and artificial intelligence.

neuroscience↗

bioRxiv2025

Attention Improves Population Codes by Warping Neural Manifolds in Human Visual Cortex

Decades of research on visual attention have revealed its numerous effects on neural responses. Two competing models have been proposed for how these effects lead to improved population representations: one highlights changes in neural tuning, while the other points towards changes in trial-by-trial noise correlations. Here, we develop a neural population manifold framework that interprets changes in neural responses as geometric transformations in high-dimensional neural space, allowing us to disentangle and quantify the effects of tuning changes and correlation changes induced by attention. Applying this framework to extensive measurements of cortical responses during different attentional tasks, we find that tuning changes are the primary driver of improved population representations. In contrast, correlation changes, though present, have minimal--or even detrimental--effects to information content due to its strong interactions with other changes (e.g., tuning, variability). Our results support the "tuning change" model of visual attention and demonstrate a general framework for adjudicating how different aspects of neural coding affect information processing.

neuroscience↗

bioRxiv2023

Noise reduction as a unified mechanism of perceptual learning in humans, macaques, and convolutional neural networks

Visual perceptual learning (VPL), defined as long-term improvement in a visual task, is considered a crucial tool for elucidating underlying visual and brain plasticity. However, the identification of a unified theory of VPL has long been controversial. Multiple existing models have proposed diverse mechanisms, including improved signal-to-noise ratio, changes in tuning curves, and reduction of noise correlations, as major contributors to improved neural representations associated with VPL. However, each model only accounts for specific aspects of the empirical findings, and there exists no theory that can comprehensively explain all empirical results. Here, we argue that all neural changes at single units can be conceptualized as geometric transformations of population response manifolds in a high-dimensional neural space. This approach enables conflicting major models of VPL to be quantitatively tested and compared within a unified computational theory. Following this approach, we found that changes in tuning curves and noise correlations, as emphasized by previous models, make no significant contributions to improved population representations by visual training. Instead, we identified neural manifold shrinkage due to reduced trial-by-trial neural response variability, a previously unexplored factor, as the primary mechanism underlying improved population representations. Furthermore, we showed that manifold shrinkage successfully accounts for learning effects across various domains, including artificial neural responses in deep neural networks trained on typical VPL tasks, multivariate BOLD signals in humans, and multi-unit activities in monkeys. These converging results suggest that our neural geometry theory offers a quantitative and comprehensive approach to explain a wide range of empirical results and to reconcile previously conflicting models of VPL.

neuroscience↗

bioRxiv2022

Representational geometries reveal differential effects of response correlations on population codes in neurophysiology and functional magnetic resonance imaging

Two sensory neurons usually display trial-by-trial response correlations given the repeated representations of an identical stimulus. The effects of such response correlations on population-level sensory coding have been the focal contention in computational neuroscience over the past few years. In the meantime, multivariate pattern analysis (MVPA) has been the leading analysis approach in functional magnetic resonance imaging (fMRI), but the effects of response correlations in voxel populations remain underexplored. Here, instead of conventional MVPA analysis, we calculate linear Fisher information of population responses in human visual cortex and hypothetically remove response correlations between voxels. We found that voxelwise response correlations generally enhance stimulus information, a result standing in stark contrast to the detrimental effects of response correlations reported in neurophysiological literature. By voxel-encoding modeling, we further show that these two seemingly opposite effects actually can coexist. Furthermore, we use principal component analysis to decompose stimulus information in population responses onto different principal dimensions in a high representational space. Interestingly, response correlations simultaneously reduce and enhance information on high- and low-variance principal dimensions, respectively. The relative strength of the two antagonistic effects within the same computational framework produces the apparent discrepancy in the effect of response correlations in neuronal and voxel populations. Our results suggest that multivariate fMRI data contain rich statistical structures that are directly related to sensory information representation, and the general computational framework to analyze neuronal and voxel population responses can be applied in many types of neural measurements.

neuroscience↗

bioRxiv2021

Sequential Temporal Anticipation Characterized by Neural Power Modulation and in Recurrent Neural Networks

Relevant prospective moments arise intermittently, while most of time is filled with irrelevant events, or noise, that constantly bombard our sensory systems. Thus, anticipating a few key moments necessitates disregarding what lies between the present and the future - the noise. Here, through examining how the brain and the recurrent neural networks (RNNs) anticipate a sequence of prospective moments without relying on any external timing cues, we provided a reinterpretation of temporal anticipation. We first designed a "premembering" task, where we marked three temporal locations in white noise and asked human listeners to detect a tone at one of these points. Using power modulation analyses, we investigated the memory-guided anticipatory processes in trials involving only flat noise. Our research revealed a unique neural-power modulation pattern for sequential temporal anticipation: neural power within the alpha-beta band range fluctuates over time, accurately identifying the marked locations on a sub-second scale and correlating with tone detection performance. To understand the functional roles of these neural modulations, we utilized RNNs optimized for the behavioral task. The hidden dynamics of the RNNs mirrored the neural modulations, and additional analyses and perturbations on the RNNs indicated that the neural power modulations in the alpha-beta band resulted from selective suppression of irrelevant noise periods and heightened sensitivity to anticipated temporal locations. Our behavioral, neural, and modeling findings collectively indicate that sequential temporal anticipation involves a process of dynamic gain control: anticipating a few significant moments involves actively disregarding irrelevant events that frequently occur.

neuroscience↗

bioRxiv2020

Functional reorganization of visual processing following long-term adaptation to optical defects

How we see is fundamentally limited by the eyes optics, which determine retinal image quality and constrain neural processing. Elucidating how long-term exposure to optical defects alters visual processing is vital for understanding the human brains capacity for and limits of sensory plasticity. Using adaptive optics to bypass the eyes optical aberrations, we assessed changes in visual processing in neurotypically-developed adults with keratoconus (KC)--a corneal disease causing severe optical aberrations during adulthood that cannot be fully corrected using conventional methods. As a result, KC patients are chronically exposed to degraded retinal images in their everyday life, making them an ideal model to understand how prolonged exposure to poor optical quality alters visual processing. Here, we show that when tested under similar fully-corrected optical conditions as neurotypical observers, KC patients exhibited altered contrast sensitivity, with impaired sensitivity for fine spatial details and better sensitivity for coarse spatial details. Both gains and losses in contrast sensitivity were more pronounced in patients with poorer habitual optical quality. Moreover, using an equivalent noise paradigm and a computational model of visual processing, we show that these alterations in visual processing are mediated by changes in signal enhancement of spatial frequency selective mechanisms. The present findings uncover fundamental properties of neural compensation mechanisms in response to long-term exposure to optical defects, which alter sensory processing and limit the benefits of improved optics. The outcome is a large-scale functional reorganization favoring the processing of sensory information less affected by the eyes optics. Significance statementThe eyes optics represent an intrinsic limit to human visual perception, determining the quality of retinal images. Neural adaptation optimizes the brains limited sensory processing capacity to the structure of the degraded retinal inputs, providing an exceptional quality of vision given these optical limitations. Here, we show that prolonged exposure to poor optical quality results in a functional reorganization of visual processing that favors sensory information less affected by the eyes optics. The present study helps elucidate how optical factors shape the way the brain processes visual information. Notably, the resulting adaptive neural plasticity limits the immediate perceptual benefits of optical interventions, a factor that must be taken into consideration when treating the increasing human population affected by optical defects.

neuroscience↗

bioRxiv2019

Hormone Levels Are Related to Altered Functional Connectivity in Prolactinomas

Background and ObjectiveProlactinomas may cause drastic hormone fluctuations throughout the body. It is not fully understood how endogenous hormone disorders such as prolactinomas reshape the patients brain. By employing the resting-state functional magnetic resonance imaging technique, we aimed to investigate the whole-brain functional connectivity (FC) and its relationship with hormone levels in patients with prolactinomas.\n\nMethodsUsing whole-brain and seed-based functional connectivity analyses, we compared FC metrics between 33 prolactinoma patients and 31 healthy controls matched with age, sex, and handedness. Then we performed partial correlation analysis to examine the relationship between FC metrics and hormone levels.\n\nResultsCompared to healthy controls, we found that prolactinoma patients showed significantly increased thalamocortical (visual cortex) and cerebellar-cerebral connectivity. In addition, endogenous hormone levels were positively correlated with the increased FC, and the hormone-FC relationships showed sex difference in prolactinoma patients.\n\nConclusionsOur findings are the first to reveal the altered FC patterns and sex-dependent hormone-FC relationships in prolactinoma patients, indicating the important role of hormone levels in the neural mechanism of brain reorganization and hyperactive intrinsic connections in prolactinomas.

neuroscience↗

bioRxiv2019

Understanding multivariate brain activity: evaluating the effect of voxelwise noise correlations on population codes in functional magnetic resonance imaging

Previous studies have shown that neurons exhibit trial-by-trial correlated activity and that such noise correlations (NCs) greatly impact the accuracy of population codes. Meanwhile, multivariate pattern analysis (MVPA) has become a mainstream approach in functional magnetic resonance imaging (fMRI), but it remains unclear how NCs between voxels influence MVPA performance. Here, we tackle this issue by combining voxel-encoding modeling and MVPA. We focus on a well-established form of NC, tuning-compatible noise correlation (TCNC), whose sign and magnitude are systematically related to the tuning similarity between two units. We first replicate the classical finding that TCNCs impair population codes in a standard neuronal population. We then extend our analysis to fMRI data, and show that voxelwise TCNCs do not impair and can even improve MVPA performance when TCNCs are strong or the number of voxels is large. We also confirm these results using standard information-theoretic analyses in computational neuroscience. Further computational analyses demonstrate that the discrepancy between the effect of TCNCs in neuronal and voxel populations can be explained by tuning heterogeneity and pool sizes. Our results provide a theoretical foundation to understand the effect of correlated activity on population codes in macroscopic fMRI data. Our results also suggest that future fMRI research could benefit from a closer examination of the correlational structure of multivariate responses, which is not directly revealed by conventional MVPA approaches.

neuroscience↗