The Impact of Noise on Memorability in Minds and Machines

Abstract

Some images are consistently remembered better than others, a stimulus property known as memorability, and deep neural networks (DNNs) are increasingly used to estimate it in place of human observers. A recent account proposes that memorable images are processed more efficiently. We argue that memorability can also be viewed as signal quality, and that efficient processing should therefore be robust to noise. Comparing memorability estimates from state-of-the-art (SOTA) models with human performance in continuous recognition tasks, we found that the two consistently diverged in how memorability changed as images were degraded by increasing levels of Gaussian noise and blur, such that models under-estimated high memorability images with noise, while over-estimating low memorability images with noise, a pattern resembling stochastic resonance. In contrast, noise degraded memorability estimates in humans, yet high memorability images were resistant to this effect. We discovered this divergence arose because conventional models are influenced by how noise alters their attention across images, whereas humans learn how images relate to one another. Building on these findings, we developed GraphMem, a model that estimates memorability from the relational structure among images rather than from the features of individual images. GraphMem outperformed SOTA models in estimating memorability under noise and blur and aligned most closely with human data. These findings caution against relying on conventional DNN-based memorability estimates for novel images, and suggest that modeling the relational structure in vision models yields more human-like memorability estimates.