A compressed code for memory discrimination

Abstract

The ability to discriminate similar visual stimuli has been used as an important index of memory function. This ability is widely thought to be supported by expanding the dimensionality of relevant neural codes, such that neural representations for the similar stimuli are maximally distinct, or "separated." An alternative hypothesis is that discrimination is supported by lossy compression of visual inputs, efficiently coding sensory information by discarding seemingly irrelevant details. A benefit of compression, relative to expansion, is that it allows the individual to efficiently retain fewer essential dimensions underlying stimulus variation--a process linked to higher-order visual processing--without hindering discrimination. Under the compression hypothesis, pattern separation is facilitated when more information from similar stimuli can be discarded, rather than preserving more information about distinct stimulus dimensions. We test the compression versus expansion hypotheses by predicting performance on the canonical mnemonic similarity task. First, we train neural networks to compress perceptual and semantic factors of stimuli, and measure lossiness of those representations using the mathematical framework underlying compression. Consistent with the compression hypothesis, and not the expansion hypothesis, we find that greater lossiness predicts the ease and performance of lure discrimination, particularly in later layers of convolutional neural networks shown to predict brain activity in the higher-order visual stream. We then empirically confirm these predictions across two sets of images, four behavioral datasets, and alternative metrics of lossiness. Finally, using task fMRI data, we identify signatures of lossy compression--neural dimensionality reduction and information loss--in the higher-order visual stream regions V4 and IT as well as hippocampal subregions dentate gyrus/CA3 and CA1 associated with lure discrimination performance. These results suggest lossy compression may support mnemonic discrimination behavior by discarding redundant and overlapping information.