Biology subjects

Koide-Majima, N.

Publications and source records attributed to Koide-Majima, N..

5 recordsLinked to original sources

bioRxiv2024

The Brain Tells a Story: Unveiling Distinct Representations of Semantic Content in Speech, Objects, and Stories in the Human Brain with Large Language Models

In recent studies, researchers have used large language models (LLMs) to explore semantic representations in the brain; however, they have typically assessed different levels of semantic content, such as speech, objects, and stories, separately. In this study, we recorded brain activity using functional magnetic resonance imaging (fMRI) while participants viewed 8.3 hours of dramas and movies. We annotated these stimuli at multiple semantic levels, which enabled us to extract latent representations of LLMs for this content. Our findings demonstrate that LLMs predict human brain activity more accurately than traditional language models, particularly for complex background stories. Furthermore, we identify distinct brain regions associated with different semantic representations, including multi-modal vision-semantic representations, which highlights the importance of modeling multi-level and multi-modal semantic representations simultaneously. We will make our fMRI dataset publicly available to facilitate further research on aligning LLMs with human brain function. Please check out our webpage at https://sites.google.com/view/llm-and-brain/.

neuroscience↗

bioRxiv2023

Mental image reconstruction from human brain activity

Visual images perceived by humans can be reconstructed from their brain activity. However, the visualization (externalization) of mental imagery remains a challenge. In this study, we demonstrated that the visual image reconstruction method proposed in the seminal study by Shen et al. (2019) heavily relied on low-level visual information decoded from the brain and could not efficiently utilize semantic information that would be recruited during mental imagery. To address this limitation, we extended the previous method to a Bayesian estimation framework and introduced the assistance of semantic information into it. Our proposed framework successfully reconstructed both seen (i.e., directly captured by the human eye) and imagined images from the brain activity. These results suggest that our framework would provide a technology for directly investigating the subjective contents of the brain.

neuroscience↗

bioRxiv2019

Topographic preservation of functional connectivity among the association areas in the human cerebral cortex

In the human sensorimotor cortex, some long-range corticocortical connections appear to preserve a fine-scale topology, in which physically close locations in the cortical region are functionally connected to physically close locations in other cortical regions. However, little is known about whether such topography preservation is unique to the sensorimotor areas or is general across other cortical areas. To investigate this question, we measured voxel-level functional connectivity using functional magnetic resonance imaging (fMRI) and visualized the fine-scale spatial organization of the connectivity patterns across the cortical surface. We found topographical preservation across regions, including the default mode network. Our results suggest that the topographical preservation of functional connectivity is not restricted to the sensorimotor cortex but also occurs in the association cortex.

neuroscience↗

bioRxiv2018

Representation of music genres based on the spectro-temporal modulation responses of the human brain

Music genre is an essential category for understanding human musical preferences and is provided based on the abstract categorization upon complex auditory stimuli. Previous neuroimaging studies have reported the involvement of the superior temporal gyrus (STG) in response to general music-related features. However, it remains largely unclear how abstract categories of music genre are represented in the brain and what acoustic features are more suited for explaining such representations. Here we examined comprehensive cortical representations and functional organization of music genres using 540 music clips. We applied a voxel-wise modeling approach to music-evoked brain activity measured using functional magnetic resonance imaging (fMRI). We observed distinct cortical organizations for different music genres in the bilateral STG, which revealed the representational relationship between various music genres, e.g., classical and hip-hop music showed opposite representations. Representations of music genres were largely explained by spectro-temporal modulation, which was modeled by a biologically plausible spectro-temporal modulation-transfer function (MTF) model. Our results elucidate the quantitative representation of music genres in the human cortex and indicate the possibility of modeling our abstract categorization of complex auditory stimuli based on the brain activity.\n\nSignificance statementMusic genre is an essential category for understanding human preferences of music. However, it is largely unknown how abstract categories of music genre are represented in the brain. Here, we examined comprehensive cortical representations of music genres by building voxel-wise models of fMRI data collected while human subjects listened to 540 music clips. We found distinct cortical organizations for various music genres in the bilateral STG. Such genre-specific cortical organization was explained by the biologically plausible MTF model. The current study elucidates the quantitative representation of music genres in the human cortex for the first time and indicates the possibility of modeling our abstract categorization of complex auditory stimuli based on the brain activity.

neuroscience↗

bioRxiv2018

Distinct dimensions of emotion in the human brain and their representation on the cortical surface

We experience a rich variety of emotions in daily life. While previous emotion studies focused on only a few predefined, restricted emotional states, a recent psychological study found a rich emotional representation in humans using a large set of diverse human-behavioural data. However, no representation of emotional states in the brain using emotion labels has been established on such a scale. To examine that, we used functional MRI to measure blood-oxygen-level-dependent (BOLD) responses when human subjects watched 3-h emotion-inducing movies labelled with 10,800 ratings regarding each of 80 emotion categories. By quantifying canonical correlations between BOLD responses and emotion ratings for the movie scenes, we found 25 significant dimensions of emotion representation in the brain. Then, we constructed a semantic space of the emotion representation and mapped the emotion categories on the cortical surface. We found that the emotion categories were smoothly represented from unimodal to transmodal regions on the cortical surface. This paper presents a cortical representation of a rich variety of emotion categories, which covers most of the emotional states suggested in traditional theories.

neuroscience↗