Biology subjects

Kanwisher, N.

Publications and source records attributed to Kanwisher, N..

15 recordsLinked to original sources

bioRxiv2026

A Unified World Model in Human Posterior Parietal Cortex

How does the brain understand the physical world, predict future events, and plan actions? Are these problems solved by a unified world model, as hypothesized by cognitive scientists and artificial intelligence researchers? Posterior parietal cortex has been implicated in all these functions, but whether the same neurons support each of them is unknown. We recorded from the superior parietal lobule (SPL) of two tetraplegic participants implanted with microelectrode arrays. SPL units preferentially engaged by intuitive physics, judging block-tower stability and viewing naturalistic physical events, also predicted whether moving objects would collide hundreds of milliseconds before contact, generalizing between different physical scenarios. These populations also encoded which hand action the participant intended to perform. These findings identify SPL as a substrate for physical world modeling.

neuroscience↗

bioRxiv2026

Zero-shot evaluations expose structured generalization limits in predictive models of the brain

Artificial neural networks (ANNs) can predict neural responses to natural images with remarkable accuracy, making them promising models of human vision. However, prediction scores alone reveal little about what computations these models have captured or how well they generalize beyond the images on which they are trained. Here, we introduce a principled zero-shot model evaluation framework in which ANN-based encoding models are frozen before being tested on not just held out images but entirely new datasets. Our strongest tests repurpose decades of cognitive neuroscience experiments as diagnostic benchmarks for identifying which computations current models capture and where they fail. To implement this framework we built encoding models of human category-selective regions (FFA, PPA, and EBA) using several pretrained ANNs and evaluated these fixed models across 7 independent fMRI datasets and 31 cognitive experiments from 10 published studies. This framework revealed three broad patterns. First, zero-shot evaluations exposed systematic limits to model predictivity that were largely hidden by standard within-dataset cross-validation. These limits were structured varying across brain regions and model classes and depending strongly on the similarity between the images used to build and test the models. Second, current models reproduced some, but not all, classical findings from cognitive neuroscience, providing a diagnosis of the computations that current models have yet to capture. Finally, models that performed well on prediction tests also tended to succeed at reproducing cognitive neuroscience findings, suggesting that prediction and explanation are closely linked. Unlike prediction scores however, cognitive tests help diagnose where and why models fail. Together, the zero-shot evaluation framework provides a scalable and unified approach for evaluating brain models across prediction and cognitive neuroscience tests, which can help us better understand what current models capture and what still remains to be explained.

neuroscience↗

bioRxiv2025

The cortical scene processing network emerges in infancy, prior to independent navigation experience

Sighted people rely on vision to recognize and navigate the local environment. By adulthood, human cortex contains at least three regions that respond selectively to visual scene information, but it remains unknown when or how these regions develop. One hypothesis is that scene selectivity emerges gradually in regions that initially prefer certain low-level visual features (e.g., peripheral visual input, high spatial frequencies, rectilinearity), and then exposure to the visual statistics of natural scenes drives the emergence of scene selective responses. However, both aspects of this hypothesis remain to be tested: how early scene selectivity first arises in human development, and whether it is driven by passive exposure to visual statistics. We therefore collected functional magnetic resonance imaging data from awake 2-9-month-old infants while they watched videos of real-world scenes with ego-motion, as well as faces, objects, and scrambled videos. We found stronger responses to scenes than control conditions in the location of all three scene regions. Scene-selective responses could not be explained by low-level visual properties of the stimuli, and were found in infants as young as 2-5 months old, with no evidence of age-related change. We also measured infants experience independently navigating (e.g., crawling), which was not necessary for the development of scene-selectivity. In sum, cortical regions are scene-selective in human infants prior to independent navigation, and after only limited exposure to visual scene statistics. Significance statementDespite extensive work on the functional organization of scene processing in the human adult visual cortex, little is known about the developmental origins of category selectivity for visual scenes. Here we used fMRI in awake human infants to discover that all three regions of the known visual scene processing system are present within the first few months of life - the youngest age yet detected. Scene-selective cortex therefore develops after only a few months of limited visual exposure, and prior to active experience using visual scene information to plan and guide independent navigation (e.g., by crawling). These findings provide a fundamental constraint on theories of how scene selectivity develops in high-level visual cortex.

neuroscience↗

bioRxiv2024

Intuitive physical reasoning is not mediated by linguistic nor exclusively domain-general abstract representations

The ability to reason about the physical world is a critical tool in the human cognitive toolbox, but the nature of the representations that mediate physical reasoning remains debated. Here, we use fMRI to illuminate this question by investigating the relationship between the physical-reasoning system and two well-characterized systems: a) the domain-general Multiple Demand (MD) system, which supports abstract reasoning, including mathematical and logical reasoning, and b) the language system, which supports linguistic computations and has been hypothesized to mediate some forms of thought. We replicate prior findings of a network of frontal and parietal areas that are robustly engaged by physical reasoning and identify an additional physical-reasoning area in the left frontal cortex, which also houses components of the MD and language systems. Critically, direct comparisons with tasks that target the MD and the language systems reveal that the physical-reasoning system overlaps with the MD system, but is dissociable from it in fine-grained activation patterns, which replicates prior work. Moreover, the physical-reasoning system does not overlap with the language system. These results suggest that physical reasoning does not rely on linguistic representations, nor exclusively on the domain-general abstract reasoning that the MD system supports.

neuroscience↗

bioRxiv2024

Representation of navigational affordances and ego-motion in the occipital place area

Humans effortlessly use vision to plan and guide navigation through the local environment, or "scene". A network of three cortical regions responds selectively to visual scene information, including the occipital (OPA), parahippocampal (PPA), and medial place areas (MPA) - but how this network supports visually-guided navigation is unclear. Recent evidence suggests that one region in particular, the OPA, supports visual representations for navigation, while PPA and MPA support other aspects of scene processing. However, most previous studies tested only static scene images, which lack the dynamic experience of navigating through scenes. We used dynamic movie stimuli to test whether OPA, PPA, and MPA represent two critical kinds of navigationally-relevant information: navigational affordances (e.g., can I walk to the left, right, or both?) and ego-motion (e.g., am I walking forward or backward? turning left or right?). We found that OPA is sensitive to both affordances and ego-motion, as well as the conflict between these cues - e.g., turning toward versus away from an open doorway. These effects were significantly weaker or absent in PPA and MPA. Responses in OPA were also dissociable from those in early visual cortex, consistent with the idea that OPA responses are not merely explained by lower-level visual features. OPA responses to affordances and ego-motion were stronger in the contralateral than ipsilateral visual field, suggesting that OPA encodes navigationally relevant information within an egocentric reference frame. Taken together, these results support the hypothesis that OPA contains visual representations that are useful for planning and guiding navigation through scenes.

neuroscience↗

bioRxiv2022

Using deep convolutional neural networks to test why human face recognition works the way it does

Human face recognition is highly accurate, and exhibits a number of distinctive and well documented behavioral "signatures" such as the use of a characteristic representational space, the disproportionate performance cost when stimuli are presented upside down, and the drop in accuracy for faces from races the participant is less familiar with. These and other phenomena have long been taken as evidence that face recognition is "special". But why does human face perception exhibit these properties in the first place? Here we use deep convolutional neural networks (CNNs) to test the hypothesis that all of these signatures of human face perception result from optimization for the task of face recognition. Indeed, as predicted by this hypothesis, these phenomena are all found in CNNs trained on face recognition, but not in CNNs trained on object recognition, even when additionally trained to detect faces while matching the amount of face experience. To test whether these signatures are in principle specific to faces, we optimized a CNN on car discrimination and tested it on upright and inverted car images. As for face perception, the car-trained network showed a drop in performance for inverted versus upright cars. Similarly, CNNs trained only on inverted faces produce an inverted inversion effect. These findings show that the behavioral signatures of human face perception reflect and are well explained as the result of optimization for the task of face recognition, and that the nature of the computations underlying this task may not be so "special" after all. Significance StatementFor decades, cognitive scientists have collected and characterized behavioral signatures of face recognition. Here we move beyond the mere curation of behavioral phenomena to asking why the human face system works the way it does. We find that many classic signatures of human face perception emerge spontaneously in CNNs trained on face discrimination, but not in CNNs trained on object classification (or on both object classification and face detection), suggesting that these long-documented properties of the human face perception system reflect optimizations for face recognition per se, not by-products of a generic visual categorization system. This work further illustrates how CNN models can be synergistically linked to classic behavioral findings in vision research, thereby providing psychological insights into human perception.

neuroscience↗

bioRxiv2021

Using child-friendly movie stimuli to study the development of face, place, and object regions from age 3 to 12 years

Scanning young children while watching short, engaging, commercially-produced movies has emerged as a promising approach for increasing data retention and quality. Movie stimuli also evoke a richer variety of cognitive processes than traditional experiments - allowing the study of multiple aspects of brain development simultaneously. However, because these stimuli are uncontrolled, it is unclear how effectively distinct profiles of brain activity can be distinguished from the resulting data. Here we develop an approach for identifying multiple distinct subject-specific Regions of Interest (ssROIs) using fMRI data collected during movie-viewing. We focused on the test case of higher-level visual regions selective for faces, scenes, and objects. Adults (N=13) were scanned while viewing a 5.5 minute child-friendly movie, as well as a traditional experiment with isolated faces, scenes, and objects. We found that just 2.7 minutes of movie data could identify subject-specific face, scene, and object regions. While successful, the movie approach was still less effective than a traditional localizer. Having validated our approach in adults, we then used the same methods on movie data collected from 3-12-year-old children (N=122). Movie response timecourses in 3-year-old childrens face, scene, and object regions were already significantly and specifically predicted by timecourses from the corresponding regions in adults. We also found evidence of continued developmental change, particularly in the face-selective posterior superior temporal sulcus. Taken together, our results reveal both early maturity and functional change in face, scene, and object regions, and more broadly highlight the promise of short, child-friendly movies for developmental cognitive neuroscience.

neuroscience↗

bioRxiv2021

Brain-like functional specialization emerges spontaneously in deep neural networks

The last quarter century of cognitive neuroscience has revealed numerous cortical regions in humans with distinct, often highly specialized functions, from recognizing faces to understanding language to thinking about what other people are thinking. But it remains unclear why the cortex exhibits this high degree of functional specialization in the first place. Here, we consider the case of face perception, using artificial neural networks to test the hypothesis that functional segregation of face recognition in the brain reflects the computational requirements of the task. We find that networks trained on generic object recognition perform poorly on face recognition and vice versa, and further that networks optimized for both tasks spontaneously segregate themselves into separate systems for faces and objects. Thus, generic visual features that suffice for object recognition are apparently suboptimal for face recognition and vice versa. We then show functional segregation to varying degrees for other visual categories, revealing a widespread tendency for optimization (without built-in task-specific inductive biases) to lead to functional specialization in machines and, we conjecture, also brains.

neuroscience↗

bioRxiv2020

Reduced language lateralization in autism and the broader autism phenotype as assessed with robust individual-subjects analyses

One of the few replicated functional brain differences between individuals with autism spectrum disorders (ASD) and neurotypical (NT) controls is reduced language lateralization. However, most prior reports relied on comparisons of group-level activation maps or functional markers that had not been validated at the individual-subject level, and/or used tasks that do not isolate language processing from other cognitive processes, complicating interpretation. Furthermore, few prior studies have examined functional responses in other functional networks, as needed to determine the selectivity of the effect. Using fMRI, we compared language lateralization between 28 ASD participants and carefully pairwise-matched controls, with the language regions defined individually with a well-validated language localizer. ASD participants showed less lateralized responses due to stronger right hemisphere activations. Further, this effect did not stem from a ubiquitous reduction in lateralization across the brain: ASD participants did not differ from controls in the lateralization of two other large-scale networks--the Theory of Mind network and the Multiple Demand network. Finally, in an exploratory study, we tested whether reduced language lateralization may also be present in NT individuals with high autistic trait load. Indeed, autistic trait load in a large set of NT participants (n=189) was associated with less lateralized language activations. These results suggest that reduced language lateralization is a robust and spatially selective neural marker of autism, present in individuals with ASD, but also in NT individuals with higher genetic liability for ASD, in line with a continuum model of underlying genetic risk.

neuroscience↗

bioRxiv2020

Cortical music selectivity does not require musical training

Recent work has shown that human auditory cortex contains neural populations anterior and posterior to primary auditory cortex that respond selectively to music. However, it is unknown how this selectivity for music arises. To test whether musical training is necessary, we measured fMRI responses to 192 natural sounds in 10 people with almost no musical training. When voxel responses were decomposed into underlying components, this group exhibited a music-selective component that was very similar in response profile and anatomical distribution to that previously seen in individuals with moderate musical training. We also found that musical genres that were less familiar to our participants (e.g., Balinese gamelan) produced strong responses within the music component, as did drum clips with rhythm but little melody, suggesting that these neural populations are broadly responsive to music as a whole. Our findings demonstrate that the signature properties of neural music selectivity do not require musical training to develop, showing that the music-selective neural populations are a fundamental and widespread property of the human brain. NEW & NOTEWORTHYWe show that music-selective neural populations are clearly present in people without musical training, demonstrating that they are a fundamental and widespread property of the human brain. Additionally, we show music-selective neural populations respond strongly to music from unfamiliar genres as well as music with rhythm but little pitch information, suggesting that they are broadly responsive to music as a whole.

neuroscience↗

bioRxiv2019

Intracranial recordings from human auditory cortex reveal a neural population selective for musical song

How are neural representations of music organized in the human brain? While neuroimaging has suggested some segregation between responses to music and other sounds, it remains unclear whether finer-grained organization exists within the domain of music. To address this question, we measured cortical responses to natural sounds using intracranial recordings from human patients and inferred canonical response components using a data-driven decomposition algorithm. The inferred components replicated many prior findings including distinct neural selectivity for speech and music. Our key novel finding is that one component responded nearly exclusively to music with singing. Song selectivity was not explainable by standard acoustic features and was co-located with speech- and music-selective responses in the middle and anterior superior temporal gyrus. These results suggest that neural representations of music are fractionated into subpopulations selective for different types of music, at least one of which is specialized for the analysis of song.

neuroscience↗

bioRxiv2019

The language network is recruited but not required for non-verbal semantic processing

The ability to combine individual meanings into complex representations of the world is often associated with language. Yet people also construct combinatorial event-level representations from non-linguistic input, e.g. from visual scenes. Here, we test whether the language network in the human brain is involved in and necessary for semantic processing of nonverbal events. In Experiment 1, we scanned participants with fMRI while they performed a semantic plausibility judgment task vs. a difficult perceptual control task on sentences and line drawings that describe/depict simple agent-patient interactions. We found that the language network responded robustly during the semantic task but not during the perceptual control task. This effect was observed for both sentences and pictures (although the response to sentences was stronger). Thus, language regions in healthy adults are engaged during a semantic task performed on pictorial depictions of events. But is this engagement necessary? In Experiment 2, we tested two individuals with global aphasia, who have sustained massive damage to perisylvian language areas and display severe language difficulties, against a group of age-matched control participants. Individuals with aphasia were severely impaired on a task of matching sentences and pictures. However, they performed close to controls in assessing the plausibility of pictorial depictions of agent-patient interactions. Overall, our results indicate that the left fronto-temporal language network is recruited but not necessary for semantic processing of nonverbal events.

neuroscience↗

bioRxiv2019

To find better neural network models of human vision, find better neural network models of primate vision

Specific deep artificial neural networks (ANNs) are the current best models of ventral visual processing and object recognition behavior in monkeys. We here explore whether models of non-human primate vision generalize to visual processing in the human primate brain. Specifically, we asked if model match to monkey IT is a predictor of model match to human IT, even when scoring those matches on different images. We found that the model match to monkey IT is a positive predictor of the model match to human IT (R = 0.36), and that this approach outperforms the current standard predictor of model accuracy on ImageNet. This suggests a more powerful approach for pre-selecting models as hypotheses of human brain processing.

neuroscience↗

bioRxiv2019

The speed of human social interaction perception

The ability to detect and understand other peoples social interactions is a fundamental part of the human visual experience that develops early in infancy and is shared with other primates. However, the neural computations underlying this ability remain largely unknown. Is the detection of social interactions a rapid perceptual process, or a slower post-perceptual inference? Here we used magnetoencephalography (MEG) decoding and computational modeling to ask whether social interactions can be detected via fast, feedforward processing. Subjects in the MEG viewed snapshots of visually matched real-world scenes containing a pair of people who were either engaged in a social interaction or acting independently. The presence versus absence of a social interaction could be read out from subjects MEG data spontaneously, even while subjects performed an orthogonal task. This readout generalized across different scenes, revealing abstract representations of social interactions in the human brain. These representations, however, did not come online until quite late, at 300 ms after image onset, well after the time period of feedforward visual processes. In a second experiment, we found that social interaction readout occurred at this same latency even when subjects performed an explicit task detecting social interactions. Consistent with these latency results, a standard feedforward deep neural network did not contain an abstract representation of social interactions at any model layer. We further showed that MEG responses distinguished between different types of social interactions (mutual gaze vs joint attention) even later, around 500 ms after image onset. Taken together, these results suggest that the human brain spontaneously extracts the presence and type of others social interactions, but does so slowly, likely relying on iterative top-down computations.

neuroscience↗

bioRxiv2018

How face perception unfolds over time

Within a fraction of a second of viewing a face, we have already determined its gender, age and identity. A full understanding of this remarkable feat will require a characterization of the computational steps it entails, along with the representations extracted at each. To this end, we used magnetoencephalography to measure the time course of neural responses to faces, thereby addressing two fundamental questions about how face processing unfolds over time. First, using representational similarity analysis, we found that facial gender and age information emerged before identity information, suggesting a coarse-to-fine processing of face dimensions. Second, identity and gender representations of familiar faces were enhanced very early on, indicating that the previously-reported behavioral benefit for familiar faces results from tuning of early feed-forward processing mechanisms. These findings start to reveal the time course of face perception in humans, and provide powerful new constraints on computational theories of face perception.

neuroscience↗