The multi-scale geometry of neural encoding

Abstract

As recorded neural populations become larger and artificial networks more powerful, there is growing interest in understanding and comparing what different populations encode. Representational geometry offers a powerful way to characterise this: it remaps stimulus space to stretch well-encoded stimulus differences while shrinking poorly encoded ones. However, this remapping can be constructed in many ways from neural responses. Here, we introduce a multi-scale representational geometry, derived from first principles, that captures how neurons encode stimulus differences at every scale. It is exactly related to mutual information, so independent of decoding assumptions. The more a stimulus difference contributes to information, the more the geometry stretches it. Applied to spatial, motor and visual codes, it shows how neurons combine to encode information, how areas differ, and which features are best encoded. A single framework thus maps the structure of encoded information, uniting geometric and information-theoretic views of neural coding.