Behavioral and causal evidence for object-based scene recognition in visual cortex

Abstract

Our visual environment can be parsed into objects and scenes, a distinction that is reflected in the organization of the human visual cortex. Previous research has shown that object and scene perception nevertheless closely interact, such that scenes influence object perception and objects influence scene perception. It remains unclear, however, whether and how objects that are not inherently diagnostic of their surroundings aid the recognition of poorly visible scenes (e.g., a person standing in a dark living room). Here, in three behavioral experiments, we show that participants made more accurate indoor/outdoor judgments when degraded scene photographs were presented together with an object than when the scene or the object was shown alone, even though the same object categories appeared in indoor and outdoor scenes. This object-driven benefit vanished once scene structure was removed through phase scrambling and was reduced when objects appeared in physically inconsistent locations within the scenes. These results suggest that objects in consistent locations (e.g., a person standing on a floor) disambiguate scene layout. Finally, in a pre-registered transcranial magnetic stimulation (TMS) study (N = 48), we provide causal evidence that the object-selective lateral occipital cortex (LOC) supports scene categorization when scene layout is disambiguated by within-scene objects. Stimulation of the LOC, particularly at 260-300 ms after stimulus onset, selectively disrupted object-based scene recognition. Together, these findings demonstrate that objects facilitate the read-out of the surrounding space in service of efficient scene recognition. Significance StatementUnderstanding how scene and object processing interact for efficient recognition is a key question in natural vision. Research has long emphasized how surrounding scenes help us identify objects, yet the reverse - how objects shape the perception of scenes - has received little attention. In the dark, does a glimpse of a floating boat tell us we are looking at a lake? In this study we demonstrate that a single object helps people recognize hardly visible scenes. This benefit required intact scene structure and depended on where the object appeared in the scene. In addition, object selective visual cortex was causally related to this benefit. Together, these findings show that objects visual appearance can be used to better understand our surroundings.