Invariance of neural representations cannot be validly inferred from neuroimaging decoding studies: Empirical and computational evidence

Abstract

Many research questions in sensory neuroscience involve determining whether the neural representation of a stimulus property is invariant or specific to a particular stimulus context (e.g., Is object representation invariant to translation? Is the representation of a face feature specific to the context of other face features?). Between these two extremes, representations may also be context-tolerant or context-sensitive. Most neuroimaging studies have used operational tests in which a target property is inferred from a significant test against the null hypothesis of the opposite property. For example, the popular cross-classification test concludes that representations are invariant or tolerant when the null hypothesis of specificity is rejected. A recently developed neurocomputational theory provides two insights regarding such tests. First, tests against the null of context-specificity, and for the alternative of context-invariance, are prone to false positives due to the way in which the underlying neural representations are transformed into indirect measurements in neuroimaging studies. Second, jointly performing tests against the nulls of invariance and specificity allows one to reach more precise and valid conclusions about the underlying representations. Here, we provide empirical and computational evidence supporting both of these theoretical insights. In our empirical study, we use encoding of orientation and spatial position in primary visual cortex as a case study, as previous research has established that these properties are encoded in a context-sensitive way. Using fMRI decoding, we show that the cross-classification test produces false-positive conclusions of invariance, but that more valid conclusions can be reached by jointly performing tests against the null of invariance. The results of two simulations further support both of these conclusions. We conclude that more valid inferences about invariance or specificity of neural representations can be reached by jointly testing against both hypotheses, and using neurocomputational theory to guide the interpretation of results. Author SummaryMany research questions in sensory neuroscience involve determining whether the representation of a stimulus property is invariant or specific to a change in stimulus context (e.g., translation-invariant object representation; configural representation of face features). Between these two extremes, representations may also be context-tolerant or context-sensitive. Most neuroimaging research has studied invariance using operational tests, among which the most widely used in recent years is cross-classification. We provide evidence from a functional MRI study, simulations, and theoretical results supporting two insights regarding such tests: (1) tests that seek to provide evidence for invariance (like cross-classification) have an inflated false positive rate, but (2) using complementary tests that seek evidence for context-specificity leads to more valid conclusions.