Aligning transformer circuit mechanisms to neural representations in relational reasoning

Abstract

Relational reasoning--the capacity to understand how elements relate to one another--is a defining feature of human intelligence, yet its computational basis remains unclear. Here, we combined human neuroimaging (7T fMRI) with artificial neural network modeling to identify circuit-level analogues of human reasoning computations. Using the Latin Square Task, we found that humans and transformers were able to generalize the task reliably, while standard architectures used in cognitive neuroscience could not. Analysing the transformer components revealed distinct computational roles: positional encoding captured the spatial structure of the task and aligned with representations in visual cortex, whereas attention encoded relational structure and mapped onto frontoparietal and default-mode networks. Attention weights tracked the relational complexity of the task, providing a computational analogue of reasoning demands. These results advance knowledge on the core algorithmic computations supporting complex reasoning, highlighting attention-based architectures as powerful models for investigating the neural and computational basis of higher cognition.