Bridging tuning and invariance with equivariant neuronal representations

Abstract

As we move through the world, we see the same visual scene from different perspectives. But how does the brain encode scene identity invariant to perspective, while remaining sensitive to these transformations? We propose a solution through equivariance, where perspective transformations induce structured changes in neuronal population responses. This framework implies a decomposition of population responses into orthogonal subspaces that are tuned and invariant. Testing our framework with large-scale neuronal recordings across four mouse visual cortical areas, we find that the equivariant structure is more pronounced in some higher-order areas (LM, AL) than in other areas (V1, RL). This equivariant structure accounts for the observed simultaneous increase in both population tuning and invariance. In comparison, early layers of an artificial neural network trained on image classification show similar structure, but later layers increase invariance at the cost of tuning. These results suggest equivariance is a principle to achieve flexible computations with neuronal populations.