Biology subjects

Fleming, R. W.

Publications and source records attributed to Fleming, R. W..

17 recordsLinked to original sources

bioRxiv2026

How to pour a cup of coffee

Pouring a drink feels deceptively trivial, yet it requires guiding a boundary-free fluid into a vessel without spilling, overflowing, or toppling it -- a task at which robots remain notoriously brittle. How humans achieve this so effortlessly is unknown, as motor control has predominantly been studied in brief, highly constrained laboratory tasks, leaving the control principles underlying ecological tasks largely unknown. Here we measured continuous sensorimotor control during liquid pouring across various containers, vessels, and speed demands. Despite substantial variation in movement trajectories and durations, individuals maintained a strikingly invariant preferred fill level. Counterintuitively, fill level variability decreased at higher fill levels, and precision was maintained even under time pressure. A stochastic optimal control model combining a data-driven nonlinear approximation of flow dynamics with a cost that balanced individualised fill level, energy expenditure and flow-rate reproduced the behaviour. Humans thus pour optimally, given their sensorimotor limits and idiosyncratic notion of "full".

neuroscience↗

bioRxiv2026

Humans use optimal eye movements to facilitate mental rotation of objects

The ability to mentally rotate objects is a fundamental feature of human cognition, and humans can use this ability to make choices about objects based on their geometry. However, remarkably little is known about how such choices are reached, and what sort of visual information might facilitate them. We devised an experiment where participants had to mentally simulate an objects rotation to choose which of two objects was better for a subsequent task, based on its shape alone. We also tracked their gaze while they made their choice, to see which visual information they were using to facilitate this mental simulation. We found that participants were consistently able to choose the most suitable object for the task, and, remarkably, the visual information they sampled was directly linked to their choices. Put simply, participants made better choices when they looked at more informative regions of the objects, and participants who sampled regions that were better for facilitating mental simulation made better choices overall. These findings reveal a direct link between fixations, simulation, and decision-making, suggesting that to perform any fine-grained mental simulation people need to direct their gaze at specific, informative points of an object to simulate its two-dimensional proximal image displacement.

animal behavior and cognition↗

bioRxiv2026

How we draw and recognize things that don't exist

How do we make sense of something weve never seen before? Classifying objects into superordinate classes like animal is a key step in interpreting novel experiences, but is challenging because radically different items (e.g., octopus, rabbit) must somehow be grouped together. In general, no single feature is shared by all members. We reasoned that to classify or imagine novel items from outside the distribution of previous experiences, observers parse objects into meaningful component features that they can mentally recombine ( compositionality). To test this, we asked participants to draw familiar and novel members of nine superordinate object classes. We then asked other participants to classify the drawings, and mark and label their defining parts. We find that human classification performance is well predicted by a Bayesian classifier that optimally combines the part labels, suggesting humans can create and classify out-of-distribution experiences through a compositional generative representation of object features.

neuroscience↗

bioRxiv2025

Human gloss perception reproduced by tiny neural networks

A key goal of visual neuroscience is to explain how our brains infer object properties like colour, curvature, or gloss. Here, we used machine learning to identify computations underlying human gloss judgments from object images--traditionally considered a challenging inference. Thousands of object images of varied shapes were rendered with a Ward reflectance model under diverse lighting and viewpoints, and we crowdsourced gloss estimates for each image. Curiously, observers judgments were highly consistent with one another, yet systematically deviated from reality. We compared these responses with neural networks trained either to estimate physical reflectance ("ground-truth networks") or to reproduce human judgments ("human-like networks"). By reducing network size, we identified the minimum computations needed for each objective. While estimating physical reflectance required deep networks, shallow networks--with as few as three layers--accurately replicated human judgments. Remarkably, even a miniscule network with just a single filter could predict human judgments better than the best ground-truth network. The human-like networks also captured known gloss illusions outside the training range. These results suggest human gloss judgments rely on simpler, more general-purpose computations than previously thought, and demonstrate the power of using tiny but interpretable neural networks to uncover functional computations in human brains.

neuroscience↗

bioRxiv2024

Human shape perception spontaneously discovers the biological origin of novel, but natural, stimuli

Humans excel at categorizing objects by shape. This facility involves identifying shape features that objects have in common with other members of their class and relies--at least in part--on semantic/cognitive constructs. For example, plants sprout branches, fish grow fins, shoes are molded to our feet. Can humans parse shapes according to the processes that give shapes their key characteristics, even when such processes are hidden? To answer this, we investigated how humans perceive the shape of cells from the olfactory system of Xenopus laevis tadpoles. These objects are novel to most humans yet occur in nature and cluster into classes following their underlying biological function. We reconstructed 3D cell models through 3D-microscopy and photogrammetry, then conducted psychophysical experiments. Human participants performed two tasks: they arranged 3D-printed cell models by similarity and rated them along eight visual dimensions. Participants were highly consistent in their arrangements and ratings and spontaneously grouped stimuli to reflect the cell classes, unwittingly revealing the underlying processes shaping these forms. Our findings thus demonstrate that human perceptual organization mechanisms spontaneously parse the biological systematicities of never-before-seen, natural shapes. Integrating such human perceptual strategies into automated systems may enhance morphology-based analysis in biology and medicine.

neuroscience↗

bioRxiv2024

Does Precision Grip Research Extend to Unconstrained, Multidigit Grasping?

Most daily tasks require using our hands. Whether taking a sip from a glass or throwing a ball, we effortlessly select appropriate grasps. Yet, despite many possible hand configurations, most grasping research has focused on the finger-and-thumb precision grip. We thus questioned whether findings on precision grip hold under unconstrained grasping conditions. To test this, we compared how participants grasped 3D objects made of brass and wood, with both precision grip and unconstrained grasps. When unconstrained, participants rarely selected precision grips, favoring multi-digit grasps. Nevertheless, in both conditions, participants shifted grasps towards the objects center of mass and, when grasp factors conflicted, the variability in their selections increased, indicating greater uncertainty about the optimal strategy. Further, despite favoring multidigit grasps, participants consistently placed the thumb and index finger on the same positions on the objects, suggesting that in multidigit grasps, the additional fingers primarily provided support. Our findings thus reveal that object material affects unconstrained grasping similarly to precision grip and imply that previous precision grip research may extend to unconstrained, multidigit conditions. NEW & NOTEWORTHYMost grasping research focuses on two-digit precision grips, yet humans have more than two fingers. Here, we test whether previous precision grip findings apply to unconstrained grasping. We find that participants often use more than two digits when free to choose but consistently place thumb and index finger similarly on objects regardless of the number of fingers used. Our results thus highlight how the large body of precision grip literature can extend to multidigit grasping.

neuroscience↗

bioRxiv2023

A simple optical flow model explains why certain object viewpoints are special

A core challenge in perception is recognizing objects across the highly variable retinal input that occurs when objects are viewed from different directions (e.g., front vs side views). It has long been known that certain views are of particular importance, but it remains unclear why. We reasoned that characterising the computations underlying visual comparisons between objects could explain the privileged status of certain qualitatively special views. We measured pose discrimination for a wide range of objects, finding large variations in performance depending on the object and the view angle, with front and back views yielding particularly good discrimination. Strikingly, a simple and biologically plausible computational model based on measuring the projected 3D optical flow between views of objects accurately predicted both successes and failures of discrimination performance. This provides a computational account of why certain views have a privileged status. Significance statementSome viewpoints of objects are qualitatively and perceptually special, making them easier to recognize and remember. We show that qualitatively special viewpoints of familiar and novel 3D objects can be predicted by an optical-flow model that measures how points on the surface shift in the image as viewpoint changes. This provides a quantitative account for why some viewpoints of objects are perceptually special.

neuroscience↗

bioRxiv2023

Efficient Visual Heuristics in the Perception of Physical Object Properties

A key to interacting with the physical world is the ability to visually infer object properties, like elasticity, allowing us to anticipate object behavior. Such perceptual inferences continue to challenge AI systems, highlighting the complexity of the underlying computations. How does the human brain solve this task? Here, we propose a resource-rational model based on learned statistics of object motion to explain how humans judge elasticity. We created 100,000 physics-based simulations of bouncing cubes with different elasticities and found that even tiny changes in initial conditions (e.g., orientation) yield starkly different trajectories. Yet, across these simulations, we identified 23 motion features that capture natural variations in elasticity. Although a weighted combination of these features reliably predicts physical elasticity, surprisingly, humans do not seem to employ cue combination when judging elasticity. Instead, they switch between different cues. A series of experiments designed to carefully tease apart several competing heuristics, suggests that observers switch between different computationally efficient yet informative cues depending on the information available in the stimulus.

neuroscience↗

bioRxiv2023

Distinct neural components of visually guided grasping during planning and execution

Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actors body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors: grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral-stream areas during grasp planning, then in premotor regions during grasp execution. Object mass was encoded in ventral-stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects. Significance StatementGrasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

neuroscience↗

bioRxiv2022

Color and gloss constancy under diverse lighting environments

When we look at an object, we simultaneously see how glossy or matte it is, how light or dark, and what color. Yet, at each point on the objects surface, both diffuse and specular reflections are mixed in different proportions, resulting in substantial spatial chromatic and luminance variations. To further complicate matters, this pattern changes radically when the object is viewed under different lighting conditions. The purpose of this study was to simultaneously measure our ability to judge color and gloss using an image set capturing diverse object and illuminant properties. Participants adjusted the hue, lightness, chroma, and specular reflectance of a reference object so that it appeared to be made of the same material as a test object. Critically, the two objects were presented under different lighting environments. We found that hue matches were highly accurate, except for under a chromatically atypical illuminant. Chroma and lightness constancy were generally poor, but these failures correlated well with simple image statistics. Gloss constancy was particularly poor, and these failures were only partially explained by reflection contrast. Importantly, across all measures, participants were highly consistent with one another in their deviations from constancy. Although color and gloss constancy hold well in simple conditions, the variety of lighting and shape in the real world presents significant challenges to our visual systems ability to judge intrinsic material properties.

neuroscience↗

bioRxiv2022

A method to estimate contact regions between hands and objects during human multi-digit grasping

In order to grasp an object successfully, we must select appropriate contact regions for our hands on the surface of the object. However, identifying such regions is challenging. Here, we describe a workflow to estimate contact regions from marker-based tracking data. Participants grasp real objects, while we track the 3D position of both the objects and the hand including the fingers joints. We first determine joint Euler angles from a selection of tracked markers positioned on the back of the hand. Then, we use state-of-the-art hand mesh reconstruction algorithms to generate a mesh model of the participants hand in the current pose and 3D position. Using objects that were either 3D printed, or 3D scanned--and are thus available as both real objects and mesh data--allows us to co-register the hand and object meshes. In turn, this allows us to estimate approximate contact regions by calculating intersections between the hand mesh and the co-registered 3D object mesh. The method may be used to estimate where and how humans grasp objects under a variety of conditions. Therefore, the method could be of interest to researchers studying visual and haptic perception, motor control, human-computer interaction in virtual and augmented reality, and robotics. SUMMARYWhen we grasp an object, multiple regions of the fingers and hand typically make contact with the objects surface. Reconstructing such contact regions is challenging. Here, we present a method for approximately estimating contact regions, by combining marker-based motion capture with existing deep learning-based hand mesh reconstruction.

neuroscience↗

bioRxiv2021

One shot generalization in humans revealed through a drawing task

Humans have the striking ability to learn and generalize new visual concepts from just a single exemplar. We suggest that when presented with a novel object, observers identify its significant features and infer a generative model of its shape, allowing them to mentally synthesize plausible variants. To test this, we showed participants abstract 2D shapes ( Exemplars) and asked them to draw new objects ( Variations) belonging to the same class. We show that this procedure created genuine novel categories. In line with our hypothesis, particular features of each Exemplar were preserved in its Variations and there was striking agreement between participants about which shape features were most distinctive. Also, we show that strategies to create Variations were strongly driven by part structure: new objects typically modified individual parts (e.g., limbs) of the Exemplar, often preserving part order, sometimes altering it. Together, our findings suggest that sophisticated internal generative models are key to how humans analyze and generalize from single exemplars.

neuroscience↗

bioRxiv2021

Friction is preferred over grasp configuration in precision grip grasping

How humans visually select where to grasp an object depends on many factors, including grasp stability and preferred grasp configuration. We examined how endpoints are selected when these two factors are brought into conflict: Do people favor stable grasps or do they prefer their natural grasp configurations? Participants reached to grasp one of three cuboids oriented so that its two corners were either aligned with, or rotated away from, each individuals natural grasp axis (NGA). All objects were made of brass (mass: 420 g) but the surfaces of their sides were manipulated to alter friction: 1) all-brass, 2) two opposing sides covered with wood, while the other two remained of brass, or 3) two opposing sides covered with sandpaper, and the two remaining brass sides smeared with vaseline. Grasps were evaluated as either clockwise (thumb to the left of finger in frontal plane) or counterclockwise of the NGA. Grasp endpoints depended on both object orientation and surface material. For the all-brass object, grasps were bimodally distributed in the NGA-aligned condition but predominantly clockwise in the NGA-unaligned condition. These data reflected participants natural grasp configuration independently of surface material. When grasping objects with different surface materials, endpoint selection changed: Participants sacrificed their usual grasp configuration to choose the more stable object sides. A model in which surface material shifts participants preferred grip angle proportionally to the perceived friction of the surfaces accounts for our results. Our findings demonstrate that a stable grasp is more important than a biomechanically comfortable grasp configuration. NEW & NOTEWORTHYWhen grasping an object, humans can place their fingers at several positions on its surface. The selection of these endpoints depends on many factors, with two of the most important being grasp stability and grasp configuration. We put these two factors in conflict and examine which is considered more important. Our results highlight that humans are not reluctant to adopt unusual grasp configurations in order to satisfy grasp stability.

neuroscience↗

bioRxiv2020

Humans can visually judge grasp quality and refine their judgments through visual and haptic feedback

How humans visually select where to grasp objects is determined by the physical object properties (e.g., size, shape, weight), the degrees of freedom of the arm and hand, as well as the task to be performed. We recently demonstrated that human grasps are near-optimal with respect to a weighted combination of different cost functions that make grasps uncomfortable, unstable or impossible e.g., due to unnatural grasp apertures or large torques. Here, we ask whether humans can consciously access these rules. We test if humans can explicitly judge grasp quality derived from rules regarding grasp size, orientation, torque, and visibility. More specifically, we test if grasp quality can be inferred (i) by using motor imagery alone, (ii) from watching grasps executed by others, and (iii) through performing grasps, i.e. receiving visual, proprioceptive and haptic feedback. Stimuli were novel objects made of 10 cubes of brass and wood (side length 2.5 cm) in various configurations. On each object, one near-optimal and one sub-optimal grasp were selected based on one cost function (e.g. torque), while the other constraints (grasp size, orientation, and visibility) were kept approximately constant or counterbalanced. Participants were visually cued to the location of the selected grasps on each object and verbally reported which of the two grasps was best. Across three experiments, participants could either (i) passively view the static objects, (ii) passively view videos of other participants grasping the objects, or (iii) actively grasp the objects themselves. Our results show that participants could already judge grasp optimality from simply viewing the objects, but were significantly better in the video and grasping session. These findings suggest that humans can determine grasp quality even without performing the grasp--perhaps through motor imagery--and can further refine their understanding of how to correctly grasp an object through sensorimotor feedback but also by passively viewing others grasp objects.

neuroscience↗

bioRxiv2020

Unsupervised Learning Predicts Human Perception and Misperception of Specular Surface Reflectance

Reflectance, lighting, and geometry combine in complex ways to create images. How do we disentangle these to perceive individual properties, like surface glossiness? We suggest that brains disentangle properties by learning to model statistical structure in proximal images. To test this, we trained unsupervised generative neural networks on renderings of glossy surfaces and compared their representations with human gloss judgments. The networks spontaneously cluster images according to distal properties such as reflectance and illumination, despite receiving no explicit information about them. Intriguingly, the resulting representations also predict the specific patterns of successes and errors in human perception. Linearly decoding specular reflectance from the models internal code predicts human gloss perception better than ground truth, supervised networks, or control models, and predicts, on an image-by-image basis, illusions of gloss perception caused by interactions between material, shape, and lighting. Unsupervised learning may underlie many perceptual dimensions in vision, and beyond.

neuroscience↗

bioRxiv2018

Object visibility, not energy expenditure, accounts for spatial biases in human grasp selection

Humans exhibit spatial biases when grasping objects. These biases may be due to actors attempting to shorten their reaching movements and therefore minimize energy expenditures. An alternative explanation could be that they arise from actors attempting to minimize the portion of a grasped object occluded from view by the hand. We re-analyze data from a recent study, in which a key condition decouples these two competing hypotheses. The analysis reveals that object visibility, not energy expenditure, most likely accounts for spatial biases observed in human grasping.

neuroscience↗

bioRxiv2018

How humans grasp three-dimensional objects

We rarely experience difficulty picking up objects, yet of all potential contact points on the surface, only a small proportion yield effective grasps. Here, we present extensive behavioral data alongside a normative model that correctly predicts human precision grasping of unfamiliar 3D objects. We tracked participants forefinger and thumb as they picked up objects of 10 wood and brass cubes configured to tease apart effects of shape, weight, orientation, and mass distribution. Grasps were highly systematic and consistent across repetitions and participants. We employed these data to construct a model which combines five cost functions related to force closure, torque, natural grasp axis, grasp aperture, and visibility. Even without free parameters, the model predicts individual grasps almost as well as different individuals predict one anothers, but fitting weights reveals the relative importance of the different constraints. The model also accurately predicts human grasps on novel 3D-printed objects with more naturalistic geometries and is robust to perturbations in its key parameters. Together, the findings provide a unified account of how we successfully grasp objects of different 3D shape, orientation, mass, and mass distribution. Author SummaryA model based on extensive behavioral data unifies the varied and fragmented literature on human grasp selection by correctly predicting human grasps across a wide variety of conditions.

neuroscience↗