Psychophysical measurement of perceived motion flow of naturalistic scenes

Abstract

The neural and computational mechanisms underlying visual motion perception have been extensively investigated over several decades, but most studies have used simple artificial stimuli such as random-dot kinematograms. Thus, it remains difficult to predict how human observers perceive optical flows in complex natural scenes. Here, we report a novel method to measure, psychophysically, optical flows perceived by human observers watching naturalistic movies, and to reveal the characteristics of human motion perception via comparison of the measured perceived flow to the ground truths and model predictions. We selected movie clips from the MPI Sintel Flow Dataset, which contains open-source computer graphics animations with ground truths. To measure the perceived vectors at a spatiotemporal point, we flashed a small dot during presentation of a brief clip and asked the observers to adjust the speed and direction of a matching random-noise stimulus, to reproduce the vector at the flashed point. The proposed method adequately estimated perceived flow, and the estimated perceived vector also indicated flow illusions, i.e., consistent deviations from the ground truths, in various ways, depending on the stimulus patterns. Comparisons with the predictions of biologically motivated models and machine vision algorithms indicated that some flow illusions were attributable to lower-level factors such as spatiotemporal pooling and signal loss, but others reflected higher-level computations including coordinate transformations that cannot be precisely predicted by existing flow estimation models. Psychophysical measurement of the optical flows that humans perceive in realistic environments constitutes a promising paradigm for advancing our understanding of visual motion perception. Significance StatementThe basic approach to studying human vision is to analyze relationships among subjective perceptual experiences, responses of neural mechanisms, and predictions of computational models. Recent technical advances have enabled researchers to access large-scale neuronal responses and model predictions for complex sensory inputs. The data on human visual perception, however, cannot be easily scaled up. Accurate measurement of rich visual experiences when viewing natural scenes remains challenging. We thus devised a novel psychophysical method to measure optical motion flows perceived by humans. We successfully visualized human-perceived flows of complex naturalistic movies and show, for the first time, ways in which human-perceived naturalistic flows agree with, and deviate from, the physical ground truths and the predictions of various visual motion models.