When the whole is less than the sum of its parts: maximum object category information and behavioral prediction in multiscale activation patterns

Abstract

Neural codes are reflected in complex, temporally and spatially specific patterns of activation. One popular approach to decode neural codes in electroencephalography (EEG) is multivariate decoding. This approach examines the discriminability of activity patterns across experimental conditions to test if EEG contains information about those conditions. However, conventional decoding analyses ignore aspects of neural activity which are informative. Specifically, EEG data can be decomposed into a large number of mathematically distinct features (e.g., entropy, Fourier and Wavelet coefficients) which can reflect different aspects of neural activity. We previously compared 30 such features of EEG data, and found that visual category, and participant behavior, can be more accurately predicted using multiscale spatiotemporally sensitive Wavelet coefficients than mean amplitude (Karimi-Rouzbahani et al., 2021b). Here, we considered that even this larger set of features may only partially capture the underlying neural code, because the brain could use a combination of encoding protocols within a single trial which is not reflected in any one mathematical feature alone. To check, we combined those mathematical features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across 3 datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed in most of the time points by the most informative individual feature (Wavelet coefficients). Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the neural code better than any combination of features. Our findings put new constraints on the models of neural information encoding in EEG.