Generative inference unifies feedback processing for learning and perception in natural and artificial vision

Abstract

We understand how neurons respond selectively to patterns in visual input to support object recognition; however, how these circuits support perceptual grouping, illusory percepts, and imagination is not understood. These perceptual experiences are thought to require combining what we have learned about the world with incoming sensory signals, yet the neural mechanism for this integration remains unclear. Here we show that networks tuned for object recognition implicitly learn the distribution of their input, which can be accessed through feedback connections that tune synaptic weights. We introduce Generative Inference, a computational framework in which feedback pathways that adjust connection weights during learning are repurposed during perception to combine learned knowledge with sensory input, fulfilling flexible inference goals such as increasing confidence. Generative Inference enables networks tuned solely for recognition to spontaneously produce perceptual grouping, illusory contours, shape completion, and pattern formation resembling imagination, while preserving their recognition abilities. The framework reproduces neural signatures observed across perceptual experiments: delayed responses in feedback-receiving layers of early visual cortex that disappear when feedback connections are disrupted. We show that, under stated assumptions, gradients of classification error approximate directions that are informative about the data distribution, establishing a theoretical connection between recognition and generation. Together, these findings show that pattern recognition and pattern generation rely on a shared computational substrate through dual use of feedback pathways. This principle explains how neural systems recognize familiar objects reliably while remaining flexible enough to interpret incomplete or ambiguous information, and suggests that reusing learning signals for perception may be a general feature of both biological brains and artificial networks.