Predictive coding narrows the gap between convolutional networks and human brain function in misspelled-word reading

Abstract

Humans can readily recognize words even when they are misspelled, though with slower responses. We investigated whether predictive coding could be a feasible computational mechanism to explain both the robustness and the additional processing time. By incorporating brain-inspired predictive coding dynamics into a convolutional neural network (CNN), we assessed whether the resulting interplay between feed-back predictions and feed-forward errors enhanced the models brain-likeness in misspelled-word reading. The initial CNN was trained to classify images of rendered text from a 1000-word Finnish vocabulary (supervised), and then enhanced with feedback predictive coding connections, which were trained with the learning objective of reconstructing the activity in the previous layer (unsupervised). The model, with and without the predictive coding dynamics enabled, was then evaluated using the same real and misspelled word stimuli that were presented to human participants during a magnetoencephalography (MEG) recording. The predictive coding dynamics improved model performance on misspelled words, particularly reducing the accuracy gap between real and word-like misspelled words, thereby aligning overall performance more closely with human behavioral patterns. Furthermore, representational similarity analysis (RSA) and multivariate regression showed a stronger correspondence between model activations and human MEG responses when predictive coding dynamics were enabled. These findings provide converging evidence for predictive coding dynamics as a biologically plausible computational mechanism for the brains ability to cope with misspelled words. Author summaryWhen we read, we can often understand a word even if it is misspelled, e.g, "lamguage". Hence, computational models of visual word recognition in the brain should also exhibit such flexibility. Convolutional neural networks (CNNs) are gaining popularity as models of visual processing in the brain, including reading, yet they typically fail when faced with misspelled words. In our study, we enhanced a CNN by adding feedback connections that perform predictive coding, i.e. continuously attempt to reconstruct the input from the initial output and adjust the output until it can do so. Through this predictive coding loop, the model continuously refines its internal representations, mimicking how the brain may "clean up" noisy inputs. We found that this predictive coding mechanism not only allowed the model to better identify the closest real word form of misspelled words, but also produced activity patterns the more closely resembling those measured from human brain using magnetoencephalography. These results suggest that predictive coding could be a key computational principle underlying the brains remarkable flexibility in reading and provide insights into how biological mechanisms could inspire brain-like computational models.