Biology subjects

Fukiage, T.

Publications and source records attributed to Fukiage, T..

3 recordsLinked to original sources

bioRxiv2025

Human-like monocular depth biases in deep neural networks

Human depth perception from 2D images is systematically distorted, yet the nature of these distortions is not fully understood. To gain insights into this fundamental problem, we compare human depth judgments with those of deep neural networks (DNNs), which have shown remarkable abilities in monocular depth estimation. Using a novel human-annotated dataset of natural indoor scenes and a systematic analysis of absolute depth judgments, we investigate error patterns in both humans and DNNs. Employing exponential-affine fitting, we decompose depth estimation errors into depth compression, per-image affine transformations (including scaling, shearing, and translation), and residual errors. Our analysis reveals that human depth judgments exhibit systematic and consistent biases, including depth compression, a vertical bias (perceiving objects in the lower visual field as closer), and consistent per-image affine distortions across participants. Intriguingly, we find that DNNs with higher accuracy partially recapitulate these human biases, demonstrating greater similarity in affine parameters and residual error patterns. This suggests that these seemingly suboptimal human biases may reflect efficient, ecologically adapted strategies for depth inference from inherently ambiguous monocular images. However, while DNNs capture metric-level residual error patterns similar to humans, they fail to reproduce human-level accuracy in ordinal depth perception within the affine-invariant space. These findings underscore the importance of evaluating error patterns beyond raw accuracy, providing new insights into how humans and computational models resolve depth ambiguity. Our dataset and methodology provide a framework for evaluating the alignment between computational models and human perceptual biases, thereby advancing our understanding of visual space representation and guiding the development of models that more faithfully capture human depth perception. Author summaryUnderstanding the characteristics of errors in depth judgments exhibited by humans and deep neural networks (DNNs) provides a foundation for developing functional models of human brain and artificial models with enhanced interpretability. To address this, we constructed a human depth judgment dataset using indoor photographs and compared human depth judgments with those of DNNs. Our results show that humans systematically compress far distances and exhibit distortions related to viewpoint shift, which remain remarkably consistent across observers. Strikingly, the better the DNNs were at depth estimation, the more they also exhibited human-like biases. This suggests that these seemingly suboptimal human biases could in fact reflect efficient strategies for inferring 3D structure from ambiguous 2D inputs. However, we also found a limit: while DNNs mimicked some human errors, they werent as good as humans at judging the relative order of objects in depth, especially when we accounted for viewpoint distortions. We believe that our dataset and discovery of multiple error factors will drive further comparative studies between humans and DNNs, facilitating model evaluations that go beyond simple accuracy to uncover how depth perception truly works--and how it might best be replicated in computational models.

neuroscience↗

bioRxiv2023

Psychophysical measurement of perceived motion flow of naturalistic scenes

The neural and computational mechanisms underlying visual motion perception have been extensively investigated over several decades, but most studies have used simple artificial stimuli such as random-dot kinematograms. Thus, it remains difficult to predict how human observers perceive optical flows in complex natural scenes. Here, we report a novel method to measure, psychophysically, optical flows perceived by human observers watching naturalistic movies, and to reveal the characteristics of human motion perception via comparison of the measured perceived flow to the ground truths and model predictions. We selected movie clips from the MPI Sintel Flow Dataset, which contains open-source computer graphics animations with ground truths. To measure the perceived vectors at a spatiotemporal point, we flashed a small dot during presentation of a brief clip and asked the observers to adjust the speed and direction of a matching random-noise stimulus, to reproduce the vector at the flashed point. The proposed method adequately estimated perceived flow, and the estimated perceived vector also indicated flow illusions, i.e., consistent deviations from the ground truths, in various ways, depending on the stimulus patterns. Comparisons with the predictions of biologically motivated models and machine vision algorithms indicated that some flow illusions were attributable to lower-level factors such as spatiotemporal pooling and signal loss, but others reflected higher-level computations including coordinate transformations that cannot be precisely predicted by existing flow estimation models. Psychophysical measurement of the optical flows that humans perceive in realistic environments constitutes a promising paradigm for advancing our understanding of visual motion perception. Significance StatementThe basic approach to studying human vision is to analyze relationships among subjective perceptual experiences, responses of neural mechanisms, and predictions of computational models. Recent technical advances have enabled researchers to access large-scale neuronal responses and model predictions for complex sensory inputs. The data on human visual perception, however, cannot be easily scaled up. Accurate measurement of rich visual experiences when viewing natural scenes remains challenging. We thus devised a novel psychophysical method to measure optical motion flows perceived by humans. We successfully visualized human-perceived flows of complex naturalistic movies and show, for the first time, ways in which human-perceived naturalistic flows agree with, and deviate from, the physical ground truths and the predictions of various visual motion models.

neuroscience↗

bioRxiv2022

Do training with blurred images make convolutional neural networks closer to humans concerning object recognition performance and internal representations?

AO_SCPLOWBSTRACTC_SCPLOWIt is suggested that experiences of perceiving blurry images in addition to sharp images contribute to the development of robust human visual processing. To computationally investigate the effect of exposure to blurry images, we trained Convolutional Neural Networks (CNNs) on ImageNet object recognition with a variety of combinations of sharp and blurry images. In agreement with related studies, mixed training on sharp and blurred images (B+S) makes the CNNs close to humans with respect to robust object recognition against a change in image blur. B+S training also reduces the texture bias of CNN in recognition of shape-texture-cue-conflict images, but the effect is not strong enough to achieve a strong shape bias comparable to what humans show. Other tests also suggest that B+S training is not sufficient to produce robust human-like object recognition based on global con-figurational features. We also show using representational similarity analysis and zero-shot transfer learning that B+S-Net does not acquire blur-robust object recognition through separate specialized sub-networks, each for sharp and blurry images, but through a single network analyzing common image features. However, blur training alone does not automatically create a mechanism like the human brain where subband information is integrated into a common representation. Our analyses suggest that experience with blurred images helps the human brain develop neural networks that robustly recognize the surrounding world, but it is not powerful enough to fill a large gap between humans and CNNs.

neuroscience↗