Human visual performance for identifying letters affected by physiologically-inspired scrambling

Abstract

AO_SCPLOWBSTRACTC_SCPLOWIn the geniculostriate pathway of the human visual system, neuronal projections carry signals from a particular retinal locus in parallel from one anatomical area to the next. Imprecision in the fidelity of these projections would place constraints on the ability of the system to perform tasks requiring positional information. We investigated the impact that "spatial scrambling" between stages would have on visual performance. We consider two stages in a simple canonical model of the early visual cortex where scrambling might occur: either the input to the first orientation-tuned mechanisms (analogous to V1 simple cells), or the output from those mechanisms. These are referred as "subcortical" (SCS) and "cortical scrambling" (CS). We developed a wavelet decomposition and resynthesis algorithm to mimic these effects, and measured human performance in letter identification affected by the two types of scrambling. Our results showed SCS and CS have distinguishable effects on both perceived noisiness of letters and letter identification threshold. Comparing human performance against a suite of pre-trained and custom convolutional neural networks (CNNs) that were trained on the scrambled stimuli, relative efficiency (calculated from the ratio of human:CNN thresholds) is higher for CS than SCS. However, in modelling human inefficiency by reducing the proportion of wavelets available to the CNNs, humans are less efficient in CS than SCS. These differences in efficiencies show humans are better at processing orientation redundant stimuli (CS) than orientation noisy stimuli (SCS). We hypothesize this reflects differences in integration properties at the input and output stages of simple cells in the cortex. Author SummaryThe brain makes sense of the input from our eyes through a system where features are extracted and combined in successive stages. Our study concerns the spatial fidelity of the connections between visual areas. Previous behavioural and physiological evidence has suggested a scrambling of neuronal projections is present in biological visual systems. In our study, we investigate the ability of the human visual system to perform letter identification with stimuli affected by different types of on-screen distortions. These distortions simulate internal scrambling occurring at two early stages in the visual hierarchy. We used convolutional neural network (CNN) models as a benchmark, against which we compared human performance to find human efficiency in handling the distortions. We found that the type of scrambling in which humans were determined to have greater "efficiency" (relative to the CNNs) depended on the analysis used. The threshold magnitude of scrambling at which the letters could no longer be identified was greater for letters scrambled after the oriented features were extracted. Conversely, when efficiency was calculated by starving the CNNs of samples until their performance declined to the human level we instead found that the effective "number of samples" used by our humans was much higher for stimuli simulating scrambling before the oriented feature stage. These differences reflect how information is pooled and combined for these two stages.