Search bioRxiv⌕ Search

Biology subjects

Yeh, L.-C.

Publications and source records attributed to Yeh, L.-C..

11 recordsLinked to original sources

Cortical alpha rhythms predictively track occluded motion trajectories

Objects in the real world frequently move along complex, non-linear trajectories shaped by their environment. As they do, they often pass temporarily out of sight, occluded by the surrounding environment. Yet humans can maintain representations of object position and motion even when objects are no longer visible. Here, we investigate the neural basis of tracking objects undergoing non-linear, environmentally constrained motion during occlusion. We recorded EEG while participants mentally tracked the position of a ball falling through differently curved pipes (Pipe condition). The pipe condition was compared to conditions in which only the ball trajectory was visible (Ball-only condition) or in which the pipe was completely covered by an occluder (Occluder condition). On occasional probe trials, the video was terminated at randomized time points and participants had to indicate the ball's last position using the mouse cursor. Behaviorally, participants systematically reported the ball ahead of its true position, with the strongest forward displacement in the occluded conditions. Neurally, multivariate decoding analysis on the EEG data revealed that alpha-band activity carried direction-specific motion information within and across conditions. Critically, cross-condition decoding revealed shared representations between perceived (Ball-only) and mentally tracked (Occluder) motion with representations in the occluder condition anticipating later representations in the ball-only condition. Together, our behavioral and neural findings provide convergent evidence that during non-linear motion tracking under occlusion, the brain codes object motion ahead of its current position, an 'ahead of time' strategy that may underlie temporal precision when tracking objects in a dynamically changing world.

neuroscience↗

Benefits and costs of contextual associations between real-world objects in the attentional blink

The attentional blink is a well-known phenomenon illustrating the limitations of human attention: When two visual targets are presented in rapid succession, identification of the second target is often impaired. While the attentional blink is known to attenuate when targets share perceptual features or category membership, real-world objects are also linked through contextual associations, shaped by objects typically occurring within the same environments. Here, we devised an attentional blink experiment in which we orthogonally manipulated contextual and categorical relationships between the two targets while controlling for their perceptual similarity. As the key result, contextual associations facilitated identification of the second target but impaired identification of the first target. These findings suggest that contextual associations yield distinct benefits and costs for visual cognition, where enhanced attentional access to subsequent targets is traded off against increased interference between targets.

neuroscience↗

Visual search is constrained by the variability of object-category templates

Real-world visual search is often performed at the category level: we search for shoes or bags without knowing their exact features in advance. This requires categorical search templates that accommodate the inherent variability within the target category. Here, we examine how the variability in search templates across categories constrains visual search performance. We quantify template variability by measuring variability in object drawings from a large online dataset (Experiment 1) and from a controlled lab-based drawing task (Experiment 2) and in turn relate this variability to performance in categorical search. Across both experiments, higher category variability, and thus broader search templates, were associated with slower responses. Moreover, the observers most prioritized object template predicted their search performance better than other observers templates, indicating that individual differences in template variability shape visual search. Together, our findings demonstrate that naturalistic visual search is governed by structured variability across both object categories and observers.

neuroscience↗

Contextual associations impact visual search across multiple processing stages

Growing evidence indicates that visual search is influenced not only by perceptual but also by semantic information. Among such semantic influences, categorical effects have been extensively studied. By contrast, the influence of contextual associations, that is, how objects commonly co-occur within specific scene contexts, remains less understood. Here, we orthogonally manipulated contextual and categorical relationships between target and distractor objects in a visual search task, while controlling for perceptual target-distractor similarity. By applying linear regression analyses, we modelled behavioral responses and event-related potentials (ERPs) from contextual, categorical, and perceptual target-distractor similarity. We found that contextual associations guide attention by interacting with categorical information (modulating the N2pc) and ultimately shape behavioral search performance jointly with perceptual and categorical similarity. These findings demonstrate that contextual object associations can guide attention, thereby supporting efficient search in complex visual environments.

neuroscience↗

Spatiotemporal representations of contextual associations for real-world objects

In the real world, objects always appear in context. Many objects are reliably associated with a certain scene context (e.g., pots appear in kitchens) and other objects that appear in the same context (e.g., pans appear together with pots). Previous neuroimaging work suggests that such contextual associations shape the neural representation of isolated objects even in the absence of the scene context. Yet, three key questions remain unanswered: (1) How do representations of contextual associations relate to perceptual and categorical representation in visual cortex, (2) how do they emerge across time, and (3) how are they mechanistically implemented? To answer these questions, we recorded fMRI and EEG while participants (human, both sexes) viewed isolated objects stemming from two scene contexts. Multivariate pattern analysis on the neural data revealed that objects from the same context were coded more similarly than objects from different contexts in object-selective LOC and scene-selective PPA, even when systematically controlling for perceptual and categorical similarities. Such contextual relation representations emerged relatively late during visual processing (i.e., after perceptual and categorical representations), specifically in the anterior PPA, and likely through a mixture of object-to-object and object-to-scene associations. Together, our results demonstrate that contextual relation representations emerged for isolated objects, and without a task that encourages their formation, suggesting that objects automatically activate context frames that support visual cognition in real-world environments.

neuroscience↗

Unpacking similarity effects in visual memory search: categorical, semantic, and visual contributions

Visual memory search involves comparing a probe item against multiple memorized items. Previous work has shown that distractor probes from a different object category than the objects in the memory set are rejected more quickly than distractor probes from the same category. Because objects belonging to the same superordinate category usually share both visual and semantic features compared with objects of different categories, it is unclear whether the category effects reported in previous studies reflected visual and/or semantic target-distractor similarity. Here, we employed old/new recognition tasks to examine the role of categorical, semantic, and visual similarity in short- and long-term memory search. Participants (N=64) performed visual long-term memory (LTM) or short-term memory (STM) search tasks involving animate and inanimate objects. Trial-wise RT variability to distractor probes in LTM and STM search was modelled using regression analyses that included predictors capturing categorical target-distractor similarity (same or different category), semantic target-distractor similarity (from a distributional semantic model), and visual target-distractor similarity (from a deep neural network). We found that categorical, semantic, and visual similarity all explained unique variance in trial-wise memory search performance. However, their respective contributions varied with memory set size and task, with STM performance being relatively more strongly influenced by visual and categorical similarity and LTM performance being relatively more strongly influenced by semantic similarity. These results clarify the nature of the representations used in memory search and reveal both similarities and differences between search in STM and LTM.

neuroscience↗

Cortical alpha rhythms interpolate occluded motion from natural scene context

Tracking objects as they dynamically move in and out of sight is critical for parsing our ever-changing real-world surroundings. Here, we explored how the interpolation of occluded object motion in natural scenes is mediated by top-down information flows expressed in cortical alpha rhythms. We recorded EEG while participants viewed videos of a person walking across a scene. We then used multivariate decoding on alpha-band responses to decode the direction of movement across the scene. In trials where the person was temporarily occluded, alpha dynamics interpolated the persons predicted movement. Critically, they did so in a context-dependent manner: When the scene context required the person to stop in front of an obstacle, alpha dynamics tracked the termination of motion during occlusion. As these effects were obtained with an orthogonal task at fixation, we conclude that alpha rhythms automatically interpolate occluded motion in a context-dependent way.

neuroscience↗

The neural time course of size constancy in natural scenes

Accurate real-world size perception relies on size constancy, a mechanism that integrates an objects retinal size with distance information. The neural time course of extracting pictorial distance cues from scenes and integrating them with retinal size information - a process referred to as scene-based size constancy - remains unknown. In two experiments, participants viewed objects with either large or small retinal sizes, presented at near or far distances in outdoor scene photographs, while performing an unrelated one-back task. We applied multivariate pattern analysis (MVPA) to time-resolved EEG data to decode the retinal size of large versus small objects, depending on their distance (near versus far) in the scenes. The objects were either perceptually similar in size (large-near versus small-far) or perceptually dissimilar in size (large-far versus small-near), reflecting size constancy. We found that the retinal size of objects could be decoded from 80 ms after scene onset onwards. Distance information modulated size decoding at least 120 ms later: from 200 ms after scene onset when objects were fixated, and from 280 ms when objects were viewed in the periphery. These findings reveal the neural time course of size constancy based on pictorial distance cues in natural scenes.

neuroscience↗

Category-based attention facilitates memory search

We often need to decide whether the object we look at is also the object we look for. When we look for one specific object, this process can be facilitated by preparatory feature-based attention. However, when we look for multiple objects at the same time (e.g., the products on our shopping list) such a strategy may no longer be possible, as research has shown that we can actively prepare to detect only one object at a time. Therefore, looking for multiple objects may additionally involve search in long-term memory, slowing down decision making. Interestingly, however, previous research has shown that memory search can be very efficient when distractor objects are from a different category than the items in the memory set. Here, using EEG, we show that this efficiency is supported by top-down attention at the category level. In Experiment 1, human participants (both sexes) performed a memory search task on individually presented objects of the same or different category as the objects in the memory set. We observed category-level attentional modulation of distractor processing from [~]150 ms after stimulus onset, expressed both as an evoked response modulation and as an increase in decoding accuracy of same-category distractors. In Experiment 2, memory search was performed on two concurrently presented objects. When both objects were distractors, spatial attention (indexed by the N2pc component) was directed to the object that was of the same category as the objects in the memory set. Together, these results demonstrate how attention can facilitate memory search. Significance statementWhen we are in the supermarket, we repeatedly decide whether a product we look at (e.g., a banana) is on our memorized shopping list (e.g., apples, oranges, kiwis). This requires searching our memory, which takes time. However, when the product is of an entirely different category (e.g., dairy instead of fruit), the decision can be made quickly. Here, we used EEG to show that this between-category advantage in memory search tasks is supported by top-down attentional modulation of visual processing: The visual response evoked by distractor objects was modulated by category membership, and spatial attention was quickly directed to the location of within-category (vs. between-category) distractors. These results demonstrate a close link between attention and memory.

neuroscience↗

Predicting cued and oddball visual search performance from neural representational similarity

Capacity limitations in visual tasks can be observed when the number of task-related objects increases. An influential idea is that such capacity limitations are determined by competition at the neural level: two objects that are encoded by shared neural populations interfere more in behavior (e.g., visual search) than two objects encoded by separate neural populations. However, the neural representational similarity of objects varies across brain regions and across time, raising the question of where and when competition determines task performance. Furthermore, it is unclear whether the association between neural representational similarity and task performance is common or unique across tasks. Here, we used neural representational similarity derived from fMRI, MEG, and deep neural networks (DNN) to predict performance on two visual search tasks involving the same objects and requiring the same responses but differing in instructions: cued visual search and oddball visual search. Separate groups of human participants (both sexes) viewed the individual objects in neuroimaging experiments to establish the neural representational similarity between those objects. Results showed that performance on both search tasks could be predicted by neural representational similarity throughout the visual system (fMRI), from 80 msec after onset (MEG), and in all DNN layers. Stepwise regression analysis, however, revealed task-specific associations, with unique variability in oddball search performance predicted by early/posterior neural similarity, and unique variability in cued search task performance predicted by late/anterior neural similarity. These results reveal that capacity limitations in superficially similar visual search tasks may reflect competition at different stages of visual processing. Significance StatementVisual search for target objects is slowed down by the presence of distractors, but not all distractors are equally distracting - the more similar a distractor is to the target, the more it slows down search. Here, we used fMRI, MEG, and a deep neural network to reveal where, when, and how neural similarity between targets and distractors predicts visual search performance across two search tasks: oddball visual search (locating the different-looking object) and cued visual search (locating the cued object). Results also revealed brain regions, time points, and feature levels that predicted task-unique performance. These results provide a neural basis for similarity theories of visual search and show that this neural basis differs across visual search tasks.

neuroscience↗

The time course of categorical and perceptual similarity effects in visual search

During visual search for objects (e.g., an apple), the surrounding distractor objects may share perceptual (tennis ball), categorical (banana), or both (peach) properties with the target. Previous studies showed that the perceptual similarity between target and distractor objects influences visual search. Here, we tested whether categorical target-distractor similarity also influences visual search, and how this influence depends on perceptual similarity. By orthogonally manipulating categorical and perceptual target-distractor similarity, we could investigate how and when the two similarities interactively affect search performance and neural correlates of spatial attention (N2pc) using electroencephalography (EEG). Behavioral results showed that categorical target-distractor similarity interacted with perceptual target-distractor similarity, such that the effect of categorical similarity was strongest when target and distractor objects were perceptually similar. EEG results showed that perceptual similarity influenced the early part of the N2pc (200-250 ms after stimulus onset), while categorical similarity influenced the later part (250-300 ms). Mirroring the behavioral results, categorical similarity interacted with perceptual similarity during this later time window, with categorical effects only observed for perceptually similar target-distractor pairs. Together, these results provide evidence for hierarchical processing in visual search: categorical properties influence spatial attention only when perceptual properties are insufficient to guide attention to the target. Public significance statementSearching for a target object among perceptually similar distractor objects (e.g., looking for an apple among peaches) is relatively difficult. In daily life, target and distractor objects may not only share perceptual but also non-perceptual properties, such as category membership (e.g., fruit). Here, we show that categorical similarity between target and distractor objects influences search performance, particularly when target and distractor objects are perceptually similar. Using electrophysiological recordings, we demonstrate that attentional selection is first influenced by perceptual and then by categorical information. This later categorical influence depended on perceptual similarity, being strongest for perceptually similar objects. These findings provide evidence for hierarchical processing in visual search, with categorical properties extracted after perceptual properties.

neuroscience↗