Biology subjects

Motoyoshi, I.

Publications and source records attributed to Motoyoshi, I..

5 recordsLinked to original sources

bioRxiv2024

Decoding and reconstruction of surface materials from EEG

The human visual system can easily recognize object material categories and estimate surface properties such as glossiness and smoothness. A number of psychophysical and computational studies suggest that the material perception depends on global feature statistics of the entire image at multiple processing levels. Neural representations of such global features, which is independent of precise retinotopy, may be captured even by EEG that have low spatial resolution. To test this possibility, here we measured visual evoked potentials (VEPs) for 191 natural images consisting of 20 categories of materials. We then sought to classify material categories and surface properties from the VEPs, and to reconstruct the rich phenomenological appearance of materials themselves via neural representations of global features as estimated from the VEPs. As a result, we found that material categories were correctly classified by the VEPs even at latencies of 150 ms or less. The apparent surface properties were also significantly classified within 175 ms (lightness, colorfulness, and smoothness) and after 200 ms (glossiness, hardness, and heaviness). In a subsequent reverse-correlation analysis, we further found that the VEPs at these latencies are highly correlated with low- and high-level global feature statistics of the surface images; Portilla-Simoncelli texture statistics and style information in deep convolutional neural network (dCNN), indicating that neural activities about such global features are reflected in the VEPs that enabled successful classification of materials. To demonstrate this idea more directly, we trained deep generative models (MVAE models) that reconstruct the surface image itself from the VEPs via style information (gram matrix of the dCNN output). The model successfully reconstructed realistic surface images, a part of which were nearly indistinguishable from the original images. These findings suggest that the neural representation of statistical image features, which were formed at short latencies in the visual cortex and reflected even in EEG signals, not simply enable human visual system to recognize material categories and evaluate surface properties but provides the essential basis for rich and complex phenomenological qualities of natural surfaces.

neuroscience↗

bioRxiv2023

Spatiotemporal cortical dynamics for rapid scene recognition as revealed by EEG decoding

The human visual system rapidly recognizes the categories and global properties of complex natural scenes. The present study investigated the spatiotemporal dynamics of neural signals involved in ultra-rapid scene recognition using electroencephalography (EEG) decoding. We recorded visual evoked potentials from 11 human observers for 232 natural scenes, each of which belonged to one of 13 natural scene categories (e.g., a bedroom or open country) and had three global properties (naturalness, openness, and roughness). We trained a deep convolutional classification model of the natural scene categories and global properties using EEGNet. Having confirmed that the model successfully classified natural scene categories and the three global properties, we applied Grad-CAM to the EEGNet model to visualize the EEG channels and time points that contributed to the classification. The analysis showed that EEG signals in the occipital lobes at short latencies (approximately 80[~] ms) contributed to the classifications other than roughness, whereas those in the frontal lobes at relatively long latencies ([~] 164 ms) contributed to the classification of naturalness and the individual scene category. These results suggest that different global properties are encoded in different cortical areas and with different timings, and that the encoding of scene categories shifts from the occipital to the frontal lobe over time.

neuroscience↗

bioRxiv2022

Spatial attention in perceptual decision making as revealed by response-locked classification image analysis

In many situations, humans serially sample information from many locations in an image to make an appropriate decision about a visual target. Spatial attention plays a crucial role in this serial vision process. To investigate the effect of spatial attention in such dynamic decision making, we applied a classification image (CI) analysis locked to the observers reaction time (RT). We asked human observers to detect as rapidly as possible a target whose contrast gradually increased on the left or right side of dynamic noise, with the presentation of a spatial cue. The analysis revealed a spatiotemporally biphasic profile of the CI which peaked at [~]350 ms before the observers response. We found that a valid cue presented at the target location shortened the RT and increased the overall amplitude of the CI, especially when the cue appeared 500-1250 ms before the observers response. The results were quantitatively accounted for by a simple perceptual decision mechanism that accumulates the outputs of the spatiotemporal contrast detector, whose gain is increased by sustained attention to the cued location.

neuroscience↗

bioRxiv2021

Photorealistic reconstruction of visual texture from EEG signals

Recent advances in brain decoding have made it possible to classify image categories based on neural activity. Increasing numbers of studies have further attempted to reconstruct the image itself. However, because images of objects and scenes inherently involve spatial layout information, the reconstruction usually requires retinotopically organized neural data with high spatial resolution, such as fMRI signals. In contrast, spatial layout does not matter in the perception of texture, which is known to be represented as spatially global image statistics in the visual cortex. This property of texture enables us to reconstruct the perceived image from EEG signals, which have a low spatial resolution. Here, we propose an MVAE-based approach for reconstructing texture images from visual evoked potentials measured from observers viewing natural textures such as the textures of various surfaces and object ensembles. This approach allowed us to reconstruct images that perceptually resemble the original textures with a photographic appearance. A subsequent analysis of the dynamic development of the internal texture representation in the VGG network showed that the reproductivity of texture rapidly improves at 200 ms latency in the lower layers but improves more gradually in the higher layers. The present approach can be used as a method for decoding the highly detailed impression of sensory stimuli from brain activity.

neuroscience↗

bioRxiv2021

A response-locked classification image analysis of the perceptual decision making : contrast detection

In many situations, humans make decisions based on serially sampled information through the observation of visual stimuli. To quantify the critical information used by the observer in such dynamic decision making, we here applied a classification image (CI) analysis locked to the observers reaction time (RT) in a simple detection task for a luminance target that gradually appeared in dynamic noise. We found that the response-locked CI shows a spatiotemporally biphasic weighting profile that peaked about 300 ms before the response, but this profile substantially varied depending on RT; positive weights dominated at short RTs and negative weights at long RTs. We show that these diverse results are explained by a simple perceptual decision mechanism that accumulates the output of the perceptual process as modelled by a spatiotemporal contrast detector. We discuss possible applications and the limitations of the response-locked CI analysis.

neuroscience↗