Fast learning, memorization and generalization: A computational characterization of sparse to dense hippocampal-cortical codes

Abstract

1Classic findings from neuropsychology and animal studies established the hippocampus as a key substrate for rapid learning and episodic memory, with the dentate gyrus exhibiting extreme sparse coding. Sparse coding has long been hypothesized to enable fast learning through pattern separation, enabling rapid separation of highly similar inputs. However, prior computational work has largely focused on episodic memory or simplified linear tasks, leaving open how hippocampal sparsity affects learning speed and generalization in complex tasks. Here, we present a systematic investigation of sparse coding in deep neural networks varying the sparsity level and location (layer depth) and evaluated the functional consequences for learning and generalization. We found that learning performance is maximized at a balanced sparsity level of [~]5%, matching empirical estimates of the hippocampal sparse code. Dimensionality and representational similarity analyses revealed that sparse layers promoted orthogonalization of input representations, mirroring hippocampal pattern separation that enables fast learning. Furthermore, sparsity in early layers led to fast learning only on the training set and poor generalization to a held out test set, reflecting memorization, while sparsity in later layers consistently aided generalization, providing implications for theories of hippocampal-cortical learning. Our findings demonstrate the power and tradeoffs of the hippocampal sparse code, and show how hippocampal-cortical circuits possess the computational capacity to support both fast learning and generalization, depending on where sparsity is implemented. We offer a unifying perspective on how the hippocampus works as a fast, sparse memory system and the hippocampal-cortical pathway as a mechanism for generalizable learning.