The geometry of low- and high-level perceptual spaces

Abstract

Low-level features are typically continuous (e.g., the gamut between two colors), but semantic information is often categorical (there is no corresponding gradient between dog and turtle) and hierarchical (animals live in land, water, or air). To determine the impact of these differences on cognitive representations, we characterized the geometry of perceptual spaces of five domains: a domain dominated by semantic information (animal names presented as words), a domain dominated by low-level features (colored textures), and three intermediate domains (animal images, lightly-texturized animal images that were easy to recognize, and heavily-texturized animal images that were difficult to recognize). Each domain had 37 stimuli derived from the same animal names. From 13 subjects (9F), we gathered similarity judgments in each domain via an efficient psychophysical ranking paradigm. We then built geometric models of each domain for each subject, in which distances between stimuli accounted for subjects similarity judgments and intrinsic uncertainty. Remarkably, the five domains had similar global properties: each required 5 to 7 dimensions, and a modest amount of spherical curvature provided the best fit. However, the arrangement of the stimuli within these embeddings depended on the level of semantic information: dendrograms derived from semantic domains (word, image, and lightly texturized images) were more tree-like than those from feature-dominated domains (heavily texturized images and textures). Thus, the perceptual spaces of domains along this feature-dominated to semantic-dominated gradient have a similar global geometry, but the points within the spaces shift to a tree-like organization when semantic information dominates. Significance StatementUnderstanding the nature of knowledge representation is a fundamental goal of systems neuroscience. Low-level visual features (e.g., color), form continuous domains, while semantic information is typically organized into categories and subcategories. Here, using a novel psychophysical paradigm and computational modeling strategy, we find that despite these major differences, the mental representations of these domains lie in spaces with similar overall geometry. However, within these spaces, semantic information is arranged in a more tree-like representation, and the transition to tree-like representations is relatively abrupt once semantic information becomes apparent. These findings provide insight into visual stream processing at an algorithmic level. Furthermore, they support the idea that processing along the ventral stream reflects commonalities of intrinsic cortical function.