Biology subjects

DiMattina, C.

Publications and source records attributed to DiMattina, C..

7 recordsLinked to original sources

bioRxiv2026

Textures are less comfortable to view than other natural images, consistent with cortical hyper-metabolism theory

Images considered uncomfortable to view often over-represent mid-range spatial frequencies (mid-SFs) relative to the average 1/f amplitude spectrum of natural images. A proposed mechanism for this discomfort known as the Cortical Hyper-metabolism Hypothesis (CHH) posits that such images over-activate the visual cortex, which is highly sensitive to mid-SFs. We tested a novel prediction of the CHH, namely that natural image categories like textures which over-represent mid-SFs will be less comfortable to view on average than other natural images. We indeed found that texture images were less comfortable to view than various non-textures, validating this prediction of the CHH and demonstrating its potential relevance to natural image aesthetics. Passing our images through models of early visual neurons demonstrated that the mean activity of a population of simulated log-Gabor filters resembling V1 simple cells best predicted discomfort. Manipulations which selectively altered the image phase or amplitude spectra while leaving the other intact were less comfortable than the originals, suggesting that visual comfort is not a simple function of either one of these characteristics. Our results suggest that although many factors contribute to the visual comfort of natural image categories, early visual processing may play an important role, consistent with the CHH.

neuroscience↗

bioRxiv2025

Local cues enable classification of image patches as surfaces, object boundaries, or illumination changes

To correctly parse the visual scene, one must detect edges and determine their underlying cause. Previous work has demonstrated that image-computable neural networks trained to differentiate natural shadow and occlusion edges exhibited sensitivity to boundary sharpness and texture differences. Although these models showed a strong correlation with human performance on an edge classification task, this previous study did not directly investigate whether humans actually make use of boundary sharpness and texture cues when classifying edges as shadows or occlusions. Here we directly investigated this using synthetic image patch stimuli formed by quilting together two different natural textures, allowing us to parametrically manipulate boundary sharpness, texture modulation, and luminance modulation. In a series of initial "training" experiments, observers were trained to correctly identify the cause of natural image patches taken from one of three categories (occlusion, shadow, uniform texture). In a subsequent series of "test" experiments, these same observers then classified 5 sets of synthetic boundary images defined by varying boundary sharpness, luminance modulation, and texture modulation cues using a set of novel parametric stimuli. These three visual cues exhibited strong interactions to determine categorization probabilities. For sharp edges, increasing luminance modulation made it less likely the patch would be classified as a texture and more likely it would be classified as an occlusion, whereas for blurred edges, increasing luminance modulation made it more likely the patch would be classified as a shadow. Boundary sharpness had a profound effect, so that in the presence of luminance modulation increasing sharpness decreased the likelihood of classification as a shadow and increased the likelihood of classification as an occlusion. Texture modulation had little effect on categorization, except in the case of a sharp boundary with zero luminance modulation. Results were consistent across all 5 stimulus sets, showing these effects are not due to the idiosyncrasies of the particular texture pairs. Human performance was found to be well explained by a simple linear multinomial logistic regression model defined on luminance, texture and sharpness cues, with slightly improved performance for a more complicated nonlinear model taking multiplicative parameter combinations into account. Our results demonstrate that human observers make use of the same cues as our previous machine learning models when detecting edges and determining their cause, helping us to better understand the neural and perceptual mechanisms of scene parsing.

neuroscience↗

bioRxiv2023

Second-order boundaries segment more easily when they are density-defined rather than feature-defined

Previous studies have demonstrated that density is an important perceptual aspect of textural appearance to which the visual system is highly attuned. Furthermore, it is known that density cues not only influence texture segmentation, but can enable segmentation by themselves, in the absence of other cues. A popular computational model of texture segmentation known as the "Filter-Rectify-Filter" (FRF) model predicts that density should be a second-order cue enabling segmentation. For a compound texture boundary defined by superimposing two single-micropattern density boundaries, a version of the FRF model in which different micropattern-specific channels are analyzed separately by different second-stage filters makes the prediction that segmentation thresholds should be identical in two cases: (1) Compound boundaries with an equal number of micropatterns on each side but different relative proportions of each variety (compound feature boundaries) and (2) Compound boundaries with different numbers of micropatterns on each side, but with each side having an identical number of each variety (compound density boundaries). We directly tested this prediction by comparing segmentation thresholds for second-order compound feature and density boundaries, comprised of two superimposed single-micropattern density boundaries comprised of complementary micropattern pairs differing either in orientation or contrast polarity. In both cases, we observed lower segmentation thresholds for compound density boundaries than compound feature boundaries, with identical results when the compound density boundaries were equated for RMS contrast. In a second experiment, we considered how two varieties of micropatterns summate for compound boundary segmentation. In the case where two single micro-pattern density boundaries are superimposed to form a compound density boundary, we find that the two channels combine via probability summation. By contrast, when they are superimposed to form a compound feature boundary, segmentation performance is worse than for either channel alone. From these findings, we conclude that density segmentation may rely on neural mechanisms different from those which underlie feature segmentation, consistent with recent findings suggesting that density comprises a separate psychophysical channel.

neuroscience↗

bioRxiv2022

Distinguishing shadows from surface boundaries using local achromatic cues

In order to accurately parse the visual scene into distinct surfaces, it is essential to determine whether a local luminance edge is caused by a boundary between two surfaces or a shadow cast across a single surface. Previous studies have demonstrated that local chromatic cues may help to distinguish edges caused by shadows from those caused by surface boundaries, but the information potentially available in local achromatic cues like contrast, texture, and penumbral blur remains poorly understood. In this study, we develop and analyze a large database of hand-labeled achromatic shadow edges to better understand what image properties distinguish them from occlusion edges. We find that both the highest contrast as well as the lowest contrast edges are more likely to be occlusions than shadows, extending previous observations based on a more limited image set. We also find that contrast cues alone can reliably distinguish the two edge categories with nearly 70% accuracy at 40x40 resolution. Logistic regression on a Gabor Filter bank (GFB) modeling a population of V1 simple cells separates the categories with nearly 80% accuracy, and furthermore exhibits tuning to penumbral blur. A Filter-Rectify Filter (FRF) style neural network extending the GFB model performed at better than 80% accuracy, and exhibited greater sensitivity to texture differences. Comparing the models with humans performing the same occlusion/shadow classification task using the same stimuli reveals better agreement on an image-by-image basis between human performance and the FRF model than the GFB model. Taken as a whole, the present results suggest that local achromatic cues like contrast, penumbral blur, and texture play an important role in distinguishing edges caused by shadows from those caused by surface boundaries. AUTHOR SUMMARYDistinguishing edges caused by changes in illumination from edges caused by surface boundaries is an essential computation for accurately parsing the visual scene. Previous psychophysical investigations examining the utility of various locally available cues to classify edges as shadows or surface boundaries have primarily focused on color, as surface boundaries often give rise to a change in color whereas shadows will not. However, even in grayscale images we can readily distinguish shadows from surface boundaries, suggesting an important role for achromatic cues in addition to color. We demonstrate using statistical analysis of natural shadow and occlusion edges that locally available achromatic cues can be exploited by machine classifiers to reliably distinguish these two edge categories. These classifiers exhibit sensitivity to blur and local texture differences, and exhibit reasonably good agreement with humans classifying edges as shadows or occlusion boundaries. As trichromatic vision is relatively rare in the animal kingdom, our work suggests how organisms lacking rich color vision can still exploit other cues to avoid mistaking illumination changes for surface changes.

neuroscience↗

bioRxiv2021

Interactions between luminance steps and luminance textures for boundary segmentation

In natural scenes, two adjacent surfaces may differ in mean luminance without any sharp change in luminance at their boundary, but rather due to different relative proportions of light and dark regions within each surface. We refer to such boundaries as luminance texture boundaries (LTBs), and in this study we investigate whether LTBs are segmented using different mechanisms than luminance step boundaries (LSBs). We develop a novel method to generate luminance texture boundaries from natural uniform textures, and using these natural LTB stimuli in a boundary segmentation task, we find that observers are much more sensitive to identical luminance differences which are defined by textures (LTBs) than by uniform luminance steps (LSBs), consistent with the possibility of different mechanisms. In a second and third set of experiments, we characterize observer performance segmenting natural LTBs in the presence of masking LSBs which observers are instructed to ignore. We show that although there may be some masking of LTB segmentation by LSBs, it is far less than that observed in a control experiment where both the masker and target are LSBs, and far less than that predicted by a model assuming identical mechanisms. Finally, we perform a fourth set of boundary segmentation experiments using artificial LTB stimuli comprised of differing proportions of white and black dots on opposite sides of the boundary. We find that these stimuli are also highly robust to masking by supra-threshold LSBs, consistent with our results using natural stimuli, and with our earlier studies using similar stimuli. Taken as a whole, these results suggest that the visual system contains mechanisms well suited to detecting surface boundaries that are robust to interference from luminance differences arising from luminance steps like those formed by cast shadows.

neuroscience↗

bioRxiv2020

Segmenting luminance-defined texture boundaries

Segmenting scenes into distinct surfaces is a basic visual perception task, and luminance differences between adjacent surfaces often provide an important segmentation cue. However, mean luminance differences between two surfaces may exist without any sharp change in albedo at their boundary, but rather from differences in the proportion of small light and dark areas within each surface, e.g. texture elements, which we refer to as a luminance texture boundary. Here we investigate the performance of human observers segmenting luminance texture boundaries. We demonstrate that a simple model involving a single stage of filtering cannot explain observer performance, unless it incorporates contrast normalization. Performing additional experiments in which observers segment luminance texture boundaries while ignoring super-imposed luminance step boundaries, we demonstrate that the one-stage model, even with contrast normalization, cannot explain performance. We then present a Filter-Rectify-Filter (FRF) model positing two cascaded stages of filtering, which fits our data well, and explains observers ability to segment luminance texture boundary stimuli in the presence of interfering luminance step boundaries. We propose that such computations may be useful for boundary segmentation in natural scenes, where shadows often give rise to luminance step edges which do not correspond to surface boundaries.

neuroscience↗

bioRxiv2018

Modeling second-order boundary perception: A machine learning approach

BackgroundVisual pattern detection and discrimination are essential first steps for scene analysis. Numerous human psychophysical studies have modeled visual pattern detection and discrimination by estimating linear templates for classifying noisy stimuli defined by spatial variations in pixel intensities. However, such methods are poorly suited to understanding sensory processing mechanisms for complex visual stimuli such as second-order boundaries defined by spatial differences in contrast or texture.\n\nMethodology / Principal FindingsWe introduce a novel machine learning framework for modeling human perception of second-order visual stimuli, using image-computable hierarchical neural network models fit directly to psychophysical trial data. This framework is applied to modeling visual processing of boundaries defined by differences in the contrast of a carrier texture pattern, in two different psychophysical tasks: (1) boundary orientation identification, and (2) fine orientation discrimination. Cross-validation analysis is employed to optimize model hyper-parameters, and demonstrate that these models are able to accurately predict human performance on novel stimulus sets not used for fitting model parameters. We find that, like the ideal observer, human observers take a region-based approach to the orientation identification task, while taking an edge-based approach to the fine orientation discrimination task. How observers integrate contrast modulation across orientation channels is investigated by fitting psychophysical data with two models representing competing hypotheses, revealing a preference for a model which combines multiple orientations at the earliest possible stage. Our results suggest that this machine learning approach has much potential to advance the study of second-order visual processing, and we outline future steps towards generalizing the method to modeling visual segmentation of natural texture boundaries.\n\nConclusions / SignificanceThis study demonstrates how machine learning methodology can be fruitfully applied to psychophysical studies of second-order visual processing.

neuroscience↗