Biology subjects

Tenenbaum, J. B.

Publications and source records attributed to Tenenbaum, J. B..

At least 19 recordsLinked to original sources

bioRxiv2026

A Unified World Model in Human Posterior Parietal Cortex

How does the brain understand the physical world, predict future events, and plan actions? Are these problems solved by a unified world model, as hypothesized by cognitive scientists and artificial intelligence researchers? Posterior parietal cortex has been implicated in all these functions, but whether the same neurons support each of them is unknown. We recorded from the superior parietal lobule (SPL) of two tetraplegic participants implanted with microelectrode arrays. SPL units preferentially engaged by intuitive physics, judging block-tower stability and viewing naturalistic physical events, also predicted whether moving objects would collide hundreds of milliseconds before contact, generalizing between different physical scenarios. These populations also encoded which hand action the participant intended to perform. These findings identify SPL as a substrate for physical world modeling.

neuroscience↗

bioRxiv2026

A Relational Code for Objects in Primate Frontal Cortex

When we remember a scene, we recall both the locations of individual objects and their spatial arrangement. Yet, how neurons encode this arrangement alongside individual locations is unknown. In this work, we recorded large-scale neural activity from the frontal and supplementary eye fields of monkeys performing a two-object spatial working-memory task. Using a continuum of two object configurations, we tested whether neurons encoded individual object locations or the spatial relationship between them. Many neurons had well-defined position tuning when a single object was presented. With two objects, responses spanned a continuum from retinotopic to relational tuning. Surprisingly, most units were better explained by relational tuning than retinotopic tuning. Relational and retinotopic units encoded complementary dimensions of the stimulus, and their observed mixture supported near-optimal population decoding of object locations. Relational tuning was systematically aligned with single-object receptive fields, enabling a readout that produced repulsive biases resembling those observed in behavior. Standard multi-object models such as averaging and normalization failed to reproduce the observed continuum, whereas a model integrating receptive fields over potential saccades captured it. These findings show that the frontal oculomotor cortex represents not only where objects are, but also how they are arranged with respect to one another.

neuroscience↗

bioRxiv2026

A neural circuit algorithm for action grammar

How does the brain generate sequences of discrete elements, such as actions or words, that follow an abstract rule (a grammar)? Motivated by an experiment in which monkeys learned to draw multi-object images according to grammatical rules, we trained a recurrent neural network to generate sequences Am Bn: producing shape A m times, then B n times, or vice versa. Our findings are threefold. First, the network plans the sequence by moving to an initial state encoding m and n along orthogonal directions for upcoming execution. Second, the network repeats each shape by progressing along a shape-specific manifold. Third, transitions from Am to Bn follow parallel trajectories between manifolds across values of n, thus preserving n. Each of these is an experimentally testable prediction. The network generalizes to longer novel sequences by starting further along the same orthogonal directions. These results reveal a neural circuit implementation of a symbolic algorithm for generating rule-based action sequences.

neuroscience↗

bioRxiv2026

Predictive coding video models capture dorsal parietal representations and human judgments for surfaces defined by motion

Interacting with the world relies on knowing its three-dimensional structure, yet vision receives a stream of two-dimensional images. Recovering 3D geometry from dynamic input is therefore foundational to vision, but lacks a computational account. Here we identify one, using artificial neural networks to predict human behavior and neuronal responses from the two major visual pathways in macaques. Using videos of camouflaged objects whose shape is revealed through motion, we found both pathways carry information for motion-defined surfaces, while neurons in parietal cortex most closely track what humans perceive. Only predictive coding video models--trained to fill in missing content in natural videos--reproduced both neuronal responses and human behavior. Predictive coding thus emerges as a principle that yields a brain-like geometric world model of the physical world.

neuroscience↗

bioRxiv2026

Working Memory of Multi-Object Scenes in Primate Frontal Cortex

Working memory allows primates to reason about complex scenes, yet how the brain maintains multiple objects in memory simultaneously remains unclear. Competing theories propose that objects are stored in discrete slots1,2, represented dynamically through switching3-6, or encoded by weighted combinations of single-object representations7-11. We formalized these hypotheses in terms of their quantitative predictions at the level of single neurons and tested them against densely recorded neural data from the dorsomedial frontal cortex and frontal eye field of monkeys trained to perform a novel multi-object working-memory task. Across cross-validated neural data, a Gain model, where population activity reflects weighted compositions of individual object responses, consistently outperformed Slot and Switching models. Trial-specific gain estimates predicted behavioral errors and reaction times, indicating that these latent weights capture meaningful fluctuations in memory fidelity. All results replicated in an independent dataset with different spatial configurations. Together, our work provides a rigorous framework to adjudicate a longstanding debate about how the frontal cortex retains multiple objects, identifying a weighted-sum representation as the format that best explains the neural data.

neuroscience↗

bioRxiv2026

Neural representation of action grammar structure in primate frontal cortex

Humans and other animals can solve new problems, even on the first attempt. This capacity to generate novel problem-solving behavior has been hypothesized to depend on brain mechanisms for recombining units of knowledge using systems of procedural rules, or grammars. Yet, whether and how the brain represents and implements grammars remains unclear, in large part due to the lack of neural recordings (typically performed in animal models) during grammar-based problem solving (typically studied in humans). Here, we address that gap, and in turn identify an underlying neural basis of grammatical behavior. We designed a visual-motor construction task in which macaque monkeys trace complex, often novel, geometric figures by generating a sequence of strokes, each a distinct shape, like a circle, dash, or chevron (action symbols1). Critically, sequencing was guided by a learned hierarchical action grammar AnBmCk, meaning "repeat shape A, then repeat shape B, then repeat shape C", where the number of repetitions (n, m, and k) varied across problems. Behavior was internally generated (no cues guiding stroke order), problem-directed (to construct a specific image), and exhibited zero-shot generalization (successful on the first trial for novel problems, including harder ones)--key hallmarks of grammar use. To identify underlying neural substrates, we recorded from multiple frontal cortical areas previously implicated in rule use or action sequencing. Activity during drawing encoded key structural properties of the action grammar: (i) shape index (A, B, C), (ii) abstract role (e.g., the role A independent of the specific shape), and (iii) ordinal position within a shape repeat. Moreover, all three properties were strongest in a single region, the pre-supplementary motor area (preSMA). We suggest the possibility that this conjunction of representations is a signature of an implementation of an iterative algorithm resembling a "for-loop" program: internally tracking how many repetitions remain for each shape index and switching to the next when complete. Our study establishes a paradigm for studying the neural basis of grammar-guided novel behavior and identifies in preSMA a dynamic representation of grammatical structure supporting such behavior.

neuroscience↗

bioRxiv2025

Fronto-Hippocampal Synchronization in Rapid Spatial Learning in Humans

Achieving goals in real-life situations--from fetching a glass of water to landing a dream job--often requires planning based on experience and executing a sequence of actions. Neurophysiological research in animal models has indicated that the orbitofrontal cortex (OFC) mediates relationships between memory, actions, and outcomes and the hippocampus and other medial temporal lobe (MTL) regions are known to be critical for rapid learning, but little is known about how these areas interact to support rapid learning and retrieval of goal-directed action sequences in humans. Here, we leverage a rare opportunity to investigate human OFC gamma oscillations and examine the coordination between the OFC and MTL during a continuous multi-step task that requires applying recently acquired experience to guide behavior. We used multisite intracranial electroencephalography (iEEG) recordings while participants searched for a hidden goal in an animated game to study neural activity in both brain areas during goal-directed behavior. Hippocampal ripples--brief high-frequency oscillations reflecting synchronized neuronal firing--are known to support memory consolidation during sleep, but their role during active memory retrieval and updating remains unclear. We found that OFC gamma activity was modulated by both memory demands and ripples in the hippocampus and adjacent structures. Notably, ripple-coupled OFC gamma during exploration was associated with subsequent task performance. We propose that hippocampal ripples mark a narrow window, supporting hippocampal-cortical communication required for successful goal encoding for future behaviors.

neuroscience↗

bioRxiv2025

A human brain network specialized for abstract formal reasoning

Humans stand out in the animal kingdom for their ability to reason in highly abstract ways. Using a deep-data precision fMRI approach, we identify and richly characterize a network of frontal brain areas that support abstract formal reasoning. This abstract reasoning network robustly dissociates from the domain-general Multiple Demand network--the current leading candidate substrate of fluid intelligence--as well as from three other networks supporting high-level cognition: the language network, the intuitive physical reasoning network, and the social reasoning network. Finally, the areas of this network respond robustly during both deductive and inductive reasoning, during classic matrix reasoning problems, and when solving multiplication and division problems. This network may therefore support the most abstract forms of reasoning, possibly constituting a human-specific adaptation.

neuroscience↗

bioRxiv2025

Evidence from Formal Logical Reasoning Reveals that the Language of Thought is not Natural Language

Humans are endowed with a powerful capacity for inductive and deductive logical thought: we easily form generalizations based on a few examples and draw conclusions from known premises. Humans also arguably have the most sophisticated communication system in the animal kingdom: natural language allows us to express complex and structured meanings. Some have therefore argued for a tight relationship between complex thought and language, postulating that reasoning, including logical reasoning, relies on linguistic representations. We systematically investigated the relationship between logical reasoning and language using two complementary approaches. First, we used non-invasive brain imaging (fMRI) to examine neural activity as healthy adults engaged in logical reasoning tasks. And second, we behaviorally evaluated logical abilities in individuals with extensive lesions to the language brain areas and consequent severe linguistic impairment. Our findings reveal that the language brain network is not engaged during logical reasoning, and patients with severe aphasia exhibit intact performance on logic tasks. Instead, inductive reasoning recruits the domain-general multiple demand network implicated broadly in goal-directed behaviors, whereas deductive reasoning draws on brain regions that are distinct from both the language and the multiple demand networks. Together, these results indicate that linguistic representations are neither utilized nor required for inductive or deductive logical reasoning. SignificanceWhich cognitive mechanisms allow humans to reason logically, to understand whether a conclusion follows from the premises? Are they the same ones that allow the assembly of words into structured representations? Scholars have debated for millennia whether logical reasoning is inextricably tied to natural language, or instead relies on a distinct "language of thought" (LOT). Using fMRI in healthy adults and evaluating logical ability in individuals with severe aphasia, we find that distinct neural systems support language processing vs. logical (inductive and deductive) reasoning. These results suggest that, at least in mature brains, language processing does not underpin logical inference, perhaps due to the distinct representational format of the logical LOT.

neuroscience↗

bioRxiv2025

Neural representation of action symbols in primate frontal cortex

A hallmark of intelligence is proficiency in solving new problems, including those that differ dramatically from problems seen before. Problem-solving, in turn, depends on goal-directed generation of novel ideas and behaviors1, which has been proposed to rely on internal representations of discrete units, or symbols, and processes that recombine these units into a large set of possible composite representations1-9. Although this view has been influential in formulating cognitive-level explanations of behavior, definitive evidence for a neuronal substrate of symbols has remained elusive. Here, we identify a neural population encoding action symbols--recombinable representations of discrete units of motor behavior--localized to a specific area of frontal cortex. In macaque monkeys performing a drawing-like task, we found behavioral evidence that action elements (strokes) exhibit three critical features indicating an underlying symbolic representation: (i) invariance over low-level motor parameters; (ii) categorical structure, reflecting discrete types of action; and (iii) recombination into novel sequences. In simultaneous neural recordings across motor, premotor, and prefrontal cortex, we found that planning-related population activity in ventral premotor cortex (PMv) encodes actions in a manner that, like behavior, reflects motor invariance, categorical structure, and recombination. Activity in no other recorded area exhibited these three properties of symbols. These findings reveal a neural representation of action symbols localized to PMv, and therefore identify a putative neural substrate for symbolic cognitive operations.

neuroscience↗

bioRxiv2023

Efficient Visual Heuristics in the Perception of Physical Object Properties

A key to interacting with the physical world is the ability to visually infer object properties, like elasticity, allowing us to anticipate object behavior. Such perceptual inferences continue to challenge AI systems, highlighting the complexity of the underlying computations. How does the human brain solve this task? Here, we propose a resource-rational model based on learned statistics of object motion to explain how humans judge elasticity. We created 100,000 physics-based simulations of bouncing cubes with different elasticities and found that even tiny changes in initial conditions (e.g., orientation) yield starkly different trajectories. Yet, across these simulations, we identified 23 motion features that capture natural variations in elasticity. Although a weighted combination of these features reliably predicts physical elasticity, surprisingly, humans do not seem to employ cue combination when judging elasticity. Instead, they switch between different cues. A series of experiments designed to carefully tease apart several competing heuristics, suggests that observers switch between different computationally efficient yet informative cues depending on the information available in the stimulus.

neuroscience↗

bioRxiv2022

The Neural Architecture of Theory-based Reinforcement Learning

Humans learn internal models of the environment that support efficient planning and flexible generalization in complex, real-world domains. Yet it remains unclear how such internal models are represented and learned in the brain. We approach this question within the framework of theory-based reinforcement learning, a strong form of model-based reinforcement learning in which the model is an intuitive theory - a rich, abstract, causal model of the environment built on a natural ontology of physical objects, intentional agents, relations, and goals. We used a theory-based reinforcement learning model to analyze brain data from human participants learning to play different Atari-style video games while undergoing functional MRI. Theories inferred by the theory-based model explained the signal in inferior frontal gyrus and other prefrontal areas better than several alternative models. Brain activity increased in response to theory update events in inferior frontal gyrus, occipital cortex, and fusiform gyrus, with separate learning signals for different theory components. This corresponded with a transient strengthening of theory representations in those regions. Finally, the effective connectivity pattern during theory updating suggests that information flows top-down from theory-coding regions in the prefrontal cortex to theory updating regions in occipital and temporal cortex. These results are consistent with a neural architecture in which top-down theory representations originating in prefrontal regions shape sensory predictions in visual areas, where factorized theory prediction errors are computed and in turn trigger bottom-up updates of the theory. This initial sketch provides a foundation for understanding of the neural representations and computations that support efficient theory-based reinforcement learning in complex, naturalistic environments.

neuroscience↗

bioRxiv2021

Meta-strategy learning in physical problem-solving: the effect of embodied experience

Embodied cognition suggests that our bodily experiences broadly shape our cognitive capabilities. We study how embodied experience affects the abstract physical problem-solving strategies people use in a virtual task where embodiment does not affect action capabilities. We compare how groups with different embodied experience - 25 children and 35 adults with congenital limb differences versus 45 children and 40 adults born with two hands - perform this task, and find that while there is no difference in overall competence, the groups use different cognitive strategies to find solutions. People born with limb differences think more before acting, but take fewer attempts to reach solutions. Conversely, development affects the particular actions children use, as well as their persistence with their current strategy. Our findings suggest that while development alters action choices and persistence, differences in embodied experience drive strategic changes in the acquisition of cognitive strategies for balancing acting with thinking. Statement of RelevanceTheories of embodied cognition suggest our cognitive and perceptual capabilities are shaped by how our bodies constrain our interactions with the world; however, these tasks often study short-term effects or setups where body differences impact task solutions. Here we compare the performance of children and adults with and without congenital limb differences (missing hands/upper limbs) on a virtual physical problem-solving task, where both groups had equal motor capabilities for interacting with the world. Across ages, participants with limb differences solved these tasks as proficiently as those without limb differences, but took fewer attempts to come to solutions, with more time spent thinking between attempts. This suggests that early life differences in embodied experience cause changes in individuals "cognitive strategies" for allocating time between thinking and acting.

neuroscience↗

bioRxiv2021

Invariant Representation of Physical Stability in the Human Brain

Successful engagement with the world requires the ability to predict what will happen next. Here we investigate how the brain makes the most basic prediction about the physical world: whether the situation in front of us is stable, and hence likely to stay the same, or unstable, and hence likely to change in the immediate future. Specifically, we ask if judgements of stability can be supported by the kinds of representations that have proven to be highly effective at visual object recognition in both machines and brains, or instead if the ability to determine the physical stability of natural scenes may require generative algorithms that simulate the physics of the world. To find out, we measured responses in both convolutional neural networks (CNNs) and the brain (using fMRI) to natural images of physically stable versus unstable scenarios. We find no evidence for generalizable representations of physical stability in either standard CNNs trained on visual object and scene classification (ImageNet), or in the human ventral visual pathway, which has long been implicated in the same process. However, in fronto-parietal regions previously implicated in intuitive physical reasoning we find both scenario-invariant representations of physical stability, and higher univariate responses to unstable than stable scenes. These results demonstrate abstract representations of physical stability in the dorsal but not ventral pathway, consistent with the hypothesis that the computations underlying stability entail not just pattern classification but forward physical simulation.

neuroscience↗

bioRxiv2020

Artificial Neural Networks Accurately Predict Language Processing in the Brain

The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a first systematic study taking this approach to higher-level cognition: human language processing, our species signature cognitive skill. We find that the most powerful transformer models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (fMRI, ECoG). Models neural fits ( brain score) and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain. SignificanceLanguage is a quintessentially human ability. Research has long probed the functional architecture of language processing in the mind and brain using diverse brain imaging, behavioral, and computational modeling approaches. However, adequate neurally mechanistic accounts of how meaning might be extracted from language are sorely lacking. Here, we report an important first step toward addressing this gap by connecting recent artificial neural networks from machine learning to human recordings during language processing. We find that the most powerful models predict neural and behavioral responses across different datasets up to noise levels. Models that perform better at predicting the next word in a sequence also better predict brain measurements - providing computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

neuroscience↗

bioRxiv2019

Elements of a stochastic 3D prediction engine in larval zebrafish prey capture

Many predatory animals rely on accurate sensory perception, predictive models, and precise pursuits to catch moving prey. Larval zebrafish intercept paramecia during their hunting behavior, but the precise trajectories of their prey have never been recorded in relation to fish movements in three dimensions.\n\nAs a means of uncovering what a simple organism understands about its physical world, we have constructed a 3D-imaging setup to simultaneously record the behavior of larval zebrafish, as well as their moving prey, during hunting. We show that zebrafish robustly transform their 3D displacement and rotation according to the position of their prey while modulating both of these variables depending on prey velocity. This is true for both azimuth and altitude, but particulars of the hunting algorithm in the two planes are slightly different to accommodate an asymmetric strike zone. We show that the combination of position and velocity perception provides the fish with a preferred future positional estimate, indicating an ability to project trajectories forward in time. Using computational models, we show that this projection ability is critical for prey capture efficiency and success. Further, we demonstrate that fish use a graded stochasticity algorithm where the variance around the mean result of each swim scales with distance from the target. Notably, this strategy provides the animal with a considerable improvement over equivalent noise-free strategies.\n\nIn sum, our quantitative and probabilistic modeling shows that zebrafish are equipped with a stochastic recursive algorithm that embodies an implicit predictive model of the world. This algorithm, built by a simple set of behavioral rules, allows the fish to optimize their hunting strategy in a naturalistic three-dimensional environment.

animal behavior and cognition↗

bioRxiv2019

A theory of learning to infer

Bayesian theories of cognition assume that people can integrate probabilities rationally. However, several empirical findings contradict this proposition: human probabilistic inferences are prone to systematic deviations from optimality. Puzzlingly, these deviations sometimes go in opposite directions. Whereas some studies suggest that people under-react to prior probabilities (base rate neglect), other studies find that people under-react to the likelihood of the data (conservatism). We argue that these deviations arise because the human brain does not rely solely on a general-purpose mechanism for approximating Bayesian inference that is invariant across queries. Instead, the brain is equipped with a recognition model that maps queries to probability distributions. The parameters of this recognition model are optimized to get the output as close as possible, on average, to the true posterior. Because of our limited computational resources, the recognition model will allocate its resources so as to be more accurate for high probability queries than for low probability queries. By adapting to the query distribution, the recognition model \"learns to infer.\" We show that this theory can explain why and when people under-react to the data or the prior, and a new experiment demonstrates that these two forms of under-reaction can be systematically controlled by manipulating the query distribution. The theory also explains a range of related phenomena: memory effects, belief bias, and the structure of response variability in probabilistic reasoning. We also discuss how the theory can be integrated with prior sampling-based accounts of approximate inference.

animal behavior and cognition↗

bioRxiv2018

Learning to act by integrating mental simulations and physical experiments

People can learn about the effects of their actions either by performing physical experiments or by running mental simulations. Physical experiments are reliable but risky; mental simulations are unreliable but safe. We investigate how people negotiate the balance between these strategies. Participants attempted to shoot a ball at a target, and could pay to take practice shots (physical experiments). They could also simply think (run mental simulations), but were incentivized to act quickly by paying for time. We demonstrate that the amount of thinking time and physical experiments is sensitive to trial characteristics in a way that is consistent with a model that integrates information across simulation and experimentation and decides online when to perform each.

animal behavior and cognition↗