Convolutional neural network models of the primate retina reveal adaptation to natural stimulus statistics

Abstract

Understanding the nonlinear encoding mechanisms of retinal ganglion cells (RGCs) in response to various visual stimuli presents a central challenge in neuroscience, driving the development of increasingly complex predictive models. Here, we systematically evaluate linear-nonlinear (LN) models - applying various regularization techniques - and convolutional neural networks (CNNs) of increasing depth, to predict RGC responses to white noise and natural movies. Our analysis includes publicly available datasets from marmoset and salamander retinas. We demonstrate that LN models, when equipped with appropriate inductive biases, can achieve robust predictive performance on neural responses to both white noise and natural movie stimuli. The optimal inductive biases vary substantially across datasets and stimulus types, indicating that the LN models performance is susceptible to these choices. This warrants care when using LN models as baselines: their performance is not fixed, and inappropriate design choices can lead to "unfair" comparisons. However, even in the optimal inductive bias scenario, CNNs consistently outperform LN models across conditions, confirming the advantage derived from their nonlinear representation capacity. Investigating cross-stimulus generalization, we observe that models trained on white noise generalize better to natural movies than vice versa. Notably, LN models exhibit a smaller performance gap between in-domain and cross-domain predictions compared to CNNs, suggesting that the nonlinear processing captured by CNNs is more stimulus-specific. Overall, this study provides valuable benchmarks and methodological insights for neuroscientists designing predictive models of retinal encoding.