Biology subjects

Kreiman, G.

Publications and source records attributed to Kreiman, G..

At least 19 recordsLinked to original sources

bioRxiv2025

Order-Selective Cells Tile Temporal Space and Predict Order Memory in Humans

Remembering the temporal order of events is critical for episodic memory, allowing us to link individual events into sequences. While the medial temporal lobe and prefrontal cortex are essential for this process, the underlying neural mechanisms remain poorly understood. Here we characterized the representation of order information at the level of single neurons and field potentials recorded from human neurosurgical patients watching naturalistic videos of everyday events and later recalling the order and content of the events depicted. We found order-selective cells (OSCs) in the human hippocampus, amygdala, and orbitofrontal cortex that responded selectively to specific event orders, independent of event content or absolute time. OSCs exhibited transient theta phase precession following their preferred order during both memory encoding and retrieval, the strength of which predicted participants order memory accuracy. During retrieval, OSC spike timing relative to theta varied with the relative position of their preferred event within the recalled event sequence, enabling selective retrieval of relevant events. These findings reveal a neural substrate for representing, encoding into and retrieving from memory absolute and relative ordinal relationships between discrete events. OSCs tile temporal space into discrete ordinal positions, thereby weaving episodic experiences into coherent temporal narratives.

neuroscience↗

bioRxiv2025

Single-neuron encoding of rapidly learned visual information reshapes human perception

Humans can swiftly learn to recognize visual objects after minimal exposure. Integrating new information with existing knowledge requires forming enduring neuronal representations to enable future recognition. Yet, the neuronal mechanisms in the human brain underlying such rapid perceptual changes remain unclear. We recorded single-neuron activity in occipital (OC) and medial temporal lobe (MTL) regions as participants rapidly learned to recognize degraded images. OC and MTL neurons modulated their activity to encode newly learned visual information and reshape perception. Population decoding revealed that OC neurons required additional processing time to resolve the identity of learned images, delaying neuronal responses in the MTL. Our findings indicate that OC supports recognition after rapid learning via extensive recurrent processing, potentially involving higher-order neocortical areas, without relying on feedback from MTL. These results provide mechanistic constraints for biologically plausible models of visual recognition and few-shot visual learning.

neuroscience↗

bioRxiv2024

Trial-by-trial inter-areal interactions in visual cortex in the presence or absence of visual stimulation

State-of-the-art computational models of vision largely focus on fitting trial-averaged spike counts to visual stimuli using overparameterized neural networks. However, a computational model of the visual cortex should predict the dynamic responses of neurons in single trials across different experimental conditions. In this study, we investigated trial-by-trial inter-areal interactions in the visual cortex by predicting neuronal activity in one area based on activity in another, distinguishing between stimulus-driven and non-stimulus-driven shared variability. We analyzed two datasets: calcium imaging from mouse V1 layers 2/3 and 4, and extracellular neurophysiological recordings from macaque V1 and V4. Our results show that neuronal activity can be predicted bidirectionally between L2/3 and L4 in mice, and between V1 and V4 in macaque monkeys, with the latter interaction exhibiting directional asymmetry. The predictability of neuronal responses varied with the type of visual stimulus, yet responses could also be predicted in the absence of visual stimulation. In mice, we observed a bimodal distribution of neurons, with some neurons primarily driven by visual inputs and others showing predictable activity during spontaneous activity despite lacking consistent visually evoked responses. Predictability also depended on intrinsic neuronal properties, receptive field overlap, and the relative timing of activity across areas. Our findings highlight the presence of both stimulus- and non-stimulus-related components in interactions between visual areas across diverse contexts and underscore the importance of non-visual shared variability between visual regions in both mice and macaques.

neuroscience↗

bioRxiv2024

Policy optimization emerges from noisy representation learning

Biological nervous systems learn both internal representations of the world and behavioral policies for acting within it. Motivated by growing evidence that representation learning is a fundamental principle underlying synaptic plasticity, we introduce Neural Stochastic Modulation (NSM): a theory of learning in which policy optimization emerges from reward-modulated noise layered on top of plasticity rules designed for representation learning. In NSM, reward-modulated noise shapes the steady-state weight distribution, guiding the network toward solutions that capture meaningful features while also maximizing reward. Interestingly, the evolving internal representations produced by our model mirror neural coding changes observed experimentally during task learning. Our results suggest that reward-modulated noise can serve as a minimal and biologically plausible mechanism for integrating representation and policy learning in the brain.

neuroscience↗

bioRxiv2024

The Impact of Scene Context on Visual Object Recognition: Comparing Humans, Monkeys, and Computational Models

During natural vision, we rarely see objects in isolation but rather embedded in rich and complex contexts. Understanding how the brain recognizes objects in natural scenes by integrating contextual information remains a key challenge. To elucidate neural mechanisms compatible with human visual processing, we need an animal model that behaves similarly to humans, so that inferred neural mechanisms can provide hypotheses relevant to the human brain. Here we assessed whether rhesus macaques could model human context-driven object recognition by quantifying visual object identification abilities across variations in the amount, quality, and congruency of contextual cues. Behavioral metrics revealed strikingly similar context-dependent patterns between humans and monkeys. However, neural responses in the inferior temporal (IT) cortex of monkeys that were never explicitly trained to discriminate objects in context, as well as current artificial neural network models, could only partially explain this cross-species correspondence. The shared behavioral variance unexplained by context-naive neural data or computational models highlights fundamental knowledge gaps. Our findings demonstrate an intriguing alignment of human and monkey visual object processing that defies full explanation by either brain activity in a key visual region or state-of-the-art models.

neuroscience↗

bioRxiv2024

Neuron-level prediction and noise can implement flexible reward-seeking behavior

We show that neural networks can implement reward-seeking behavior using only local predictive updates and internal noise. These networks are capable of autonomous interaction with an environment and can switch between explore and exploit behavior, which we show is governed by attractor dynamics. Networks can adapt to changes in their architectures, environments, or motor interfaces without any external control signals. When networks have a choice between different tasks, they can form preferences that depend on patterns of noise and initialization, and we show that these preferences can be biased by network architectures or by changing learning rates. Our algorithm presents a flexible, biologically plausible way of interacting with environments without requiring an explicit environmental reward function, allowing for behavior that is both highly adaptable and autonomous. Code is available at https://github.com/ccli3896/PaN.

neuroscience↗

bioRxiv2024

Invariant Neural Representation of Parts of Speech in the Human Brain

Elucidating the internal representation of language in the brain has major implications for cognitive science, brain disorders, and artificial intelligence. A pillar of linguistic studies is the notion that words have defined functions, often referred to as parts of speech. Here we recorded invasive neurophysiological responses from 1,801 electrodes in 20 patients with pharmacologically-resistant epilepsy while they were presented with two-word phrases consisting of an adjective and a noun. We observed neural signals that distinguished between these two parts of speech. The representation of parts of speech showed invariance across visual and auditory presentation modalities, robustness to word properties like length, order, frequency, and semantics, and even generalized across different languages. The selective signals were circumscribed within a small region in the left lateral orbitofrontal cortex. This selective, invariant, and localized representation of parts of speech extends classic fronto-temporal language models, providing a target for mechanistic and causal tests of how the brain represents basic building blocks of language, and introduces a systematic approach to elucidate the orchestration of more complex aspects of language.

neuroscience↗

bioRxiv2023

Behavioral relevance of category selectivity revealed by human ECoG data

Object recognition is a crucial brain function that involves a complex interplay between various brain regions. However, the behavioral relevance of functional interactions between these regions remains largely unexplored. In this study, we examined the functional interactions between different brain regions during object recognition using intracranial electrocorticography (ECoG) recordings in subjects diagnosed with pharmacologically intractable epilepsy. We computed the phase locking value (PLV) between different brain areas and its category selectivity, and assessed its behavioral relevance by comparing correctly and incorrectly performed trials. Our results revealed that phase locking between brain regions varies across different object categories and that this variability significantly influences the perceptual behavior of subjects. Importantly, we found that the behavioral relevance of these interactions is spatially organized, with long-range (global) connections being more behaviorally relevant for the frontal lobe and local connections being more crucial for the occipital lobe. These findings underscore the unique roles of different brain areas in object recognition and pave the way for more nuanced explorations of the interplay between brain regions in object recognition and other cognitive functions.

neuroscience↗

bioRxiv2023

An adversarial collaboration to critically evaluate theories of consciousness

Different theories explain how subjective experience arises from brain activity1,2. These theories have independently accrued evidence, yet, confirmation bias and dependence on design choices hamper progress in the field3. Here, we present an open science adversarial collaboration which directly juxtaposes Integrated Information Theory (IIT)4,5 and Global Neuronal Workspace Theory (GNWT)6-10, employing a theory-neutral consortium approach11,12. We investigate neural correlates of the content and duration of visual experience. The theory proponents and the consortium developed and preregistered the experimental design, divergent predictions, expected outcomes, and their interpretation12. 256 human subjects viewed suprathreshold stimuli for variable durations while neural activity was measured with functional magnetic resonance imaging, magnetoencephalography, and electrocorticography. We find information about conscious content in visual, ventro-temporal and inferior frontal cortex, with sustained responses in occipital and lateral temporal cortex reflecting stimulus duration, and content-specific synchronization between frontal and early visual areas. These results confirm some predictions of IIT and GNWT, while substantially challenging both theories: for IIT, a lack of sustained synchronization within posterior cortex contradicts the claim that network connectivity specifies consciousness. GNWT is challenged by the general lack of ignition at stimulus offset and limited representation of certain conscious dimensions in prefrontal cortex. Beyond challenging the theories themselves, we present an alternative approach to advance cognitive neuroscience through a principled, theory-driven, collaborative effort. We highlight the challenges to change peoples mind 13 and the need for a quantitative framework integrating evidence for systematic theory testing and building.

neuroscience↗

bioRxiv2023

Hippocampal Theta Phase Precession Supports Memory Formation and Retrieval of Naturalistic Experience in Humans

Linking different experiences together is a key aspect of episodic memory. A potential neural mechanism for linking sequential events over time is phase precession, which causes neurons to fire progressively earlier in time relative to theta-frequency local field potential oscillations. However, no direct link between phase precession and behaviorally assessed memory encoding or retrieval success has been established. We recorded the activity of single neurons and local field potentials in the human medial temporal lobe (MTL) while participants encoded and retrieved memories of movie clips. Transient brief theta bouts and theta phase precession were observed following cognitive boundaries during movie watching as well as following stimulus onset during memory retrieval. Phase precession was dynamic, with different neurons exhibiting phase precession in different task periods. The strength of phase precession provided information about memory encoding and retrieval success that was not available in firing rates, thereby linking the temporal code established by phase precession to behaviorally assessed memory strength. These data reveal phase precession during non spatial memory in humans and provide direct neural evidence for a functional role of phase precession in episodic memory.

neuroscience↗

bioRxiv2023

Integration of recognition, episodic, and associative memories during complex human behavior

The ability to transiently remember what happened where and when is a cornerstone of cognitive function. Forming and recalling working memories depends on detecting novelty, building associations to prior knowledge, and dynamically retrieving context-relevant information. Previous studies have scrutinized the neural machinery for individual components of recognition or associative memory under laboratory conditions, such as recalling elements from arbitrary lists of words or pictures. In this study, we implemented a well-known card- matching game that integrates multiple components of memory formation together in a naturalistic setting to investigate the dynamic neural processes underlying complex natural human memory. We recorded intracranial field potentials from 1,750 depth or subdural electrodes implanted in 20 patients with pharmacologically-intractable epilepsy while they were performing the task. We leveraged generalized linear models to simultaneously assess the relative contribution of neural responses to distinct task components. Neural activity in the gamma frequency band signaled novelty and graded degrees of familiarity, represented the strength and outcome of associative recall, and finally reflected visual feedback on a trial-by-trial basis. We introduce an attractor-based neural network model that provides a plausible first-order approximation to capture the behavioral and neurophysiological observations. The large-scale data and models enable dissociating and at the same time dynamically tracing the different cognitive components during fast, complex, and natural human memory behaviors.

neuroscience↗

bioRxiv2023

Out of sight, out of mind: Responses in primate ventral visual cortex track individual fixations during natural vision

During natural vision, primates shift their gaze several times per second with large, ballistic eye movements known as saccades. Open questions remain as to whether visual neurons retain their classical retinotopic response properties during natural vision or whether neurons integrate information across fixations and predict the consequences of impending saccades. Answers are especially wanting for vision in complex scenes relevant to natural behavior. We let 13 monkeys freely view thousands of large natural images, recorded over 883 hours of neuronal responses throughout the ventral visual pathway across 4.7 million fixations, and designed flexible analyses to reveal the spatial, temporal, and feature selectivity of the responses. Ventral visual responses followed each fixation and did not become gaze-invariant as monkeys examined an image over seconds. Computational models revealed that neuronal responses corresponded to eye-centered receptive fields. The results suggest that ventral visual cortex remains predominantly retinotopic during natural vision and does not establish a gaze-independent representation of the world.

neuroscience↗

bioRxiv2022

Integrating artificial and biological neural networks to improve animal task performance using deep reinforcement learning

Artificial neural networks have performed remarkable feats in various domains but lack the flexibility and generalization power of biological neural networks. Given their different capabilities, it would be advantageous to build systems where both network types can synergistically interact. As proof-of-principle, we show how to create such a hybrid system and harness it to improve animal performance on biologically relevant tasks. Using optogenetics, we interfaced the nervous system of the nematode Caenorhabditis elegans with a deep reinforcement learning agent, enabling the animal to navigate to targets and enhancing its food search ability. Agents adapted to strikingly different sites of neural integration and learned site-specific activations to improve performance on a target-finding task. The animal plus agent displayed cooperative computation and generalized to novel environments. This work constitutes a demonstration of how to improve task performance in animals using artificial intelligence interfaced with a nervous system.

neuroscience↗

bioRxiv2022

Periodic Time Cells in Human Entorhinal Cortex

Grid cells in the entorhinal cortex demonstrate spatially periodic firing patterns, which are thought to provide a map of space on behaviorally relevant length scales. It is unknown, however, whether such periodicity exists for behaviorally relevant time scales in the human brain. Here we investigated neuronal firing during a temporally continuous uninterrupted experience by presenting fourteen neurosurgical patients with an audiovisual video while recording single neuron activity from multiple brain regions. We report on a set of units that modulate their activity in a strikingly periodic manner across different timescales--from seconds to many minutes. These cells were most prevalent in the entorhinal cortex. Time within the video could be decoded from their population activity. Furthermore, these cells remapped their dominant periodicity to shorter timescales during a subsequent recognition memory task. When the audiovisual sequence was presented at two different speeds (regular and faster), a significant percentage of these temporally periodic cells (TPCs) maintained their timescales, suggesting a degree of invariance with respect to the narrative content. The temporal periodicity of TPCs may complement the spatial periodicity of grid cells Whether these cells provide scalable spatiotemporal metrics for encoding and retrieval of human experience warrants future investigations.

neuroscience↗

bioRxiv2022

Task-specific neural processes underlying conflict resolution during cognitive control

Cognitive control involves flexibly combining multiple sensory inputs with task-dependent goals during decision making. Several tasks have been proposed to examine cognitive control, including Stroop, Eriksen-Flanker, and the Multi-source interference task. Because these tasks have been studied independently, it remains unclear whether the neural signatures of cognitive control reflect abstract control mechanisms or specific combinations of sensory and behavioral aspects of each task. To address this question, here we recorded invasive neurophysiological signals from 16 subjects and directly compared the three tasks against each other. Neural activity patterns in the theta and high-gamma frequency bands differed between incongruent and congruent conditions, revealing strong modulation by conflicting task demands. These neural signals were specific to each task, generalizing within a task but not across tasks. These results highlight the complex interplay between sensory inputs, motor outputs, and task demands and argue against a universal and abstract representation of conflict.

neuroscience↗

bioRxiv2021

Decoding of human identity by computer vision and neuronal vision

Extracting meaning from a dynamic and variable flow of incoming information is a major goal of both natural and artificial intelligence. Computer vision (CV) guided by deep learning (DL) has made significant strides in recognizing a specific identity despite highly variable attributes1,2. This is the same challenge faced by the nervous system and partially addressed by the concept cells--neurons exhibiting selective firing in response to specific persons/places, described in the human medial temporal lobe (MTL)3-6. Yet, access to neurons representing a particular concept is limited due to these neurons sparse coding. It is conceivable, however, that the information required for such decoding is present in relatively small neuronal populations. To evaluate how well neuronal populations encode identity information in natural settings, we recorded neuronal activity from multiple brain regions of nine neurosurgical epilepsy patients implanted with depth electrodes, while the subjects watched an episode of the TV series "24". We implemented DL models that used the time-varying population neural data as inputs and decoded the visual presence of the main characters in each frame. Before training and testing the DL models, we devised a minimally supervised CV algorithm (with comparable performance against manually-labelled data7) to detect and label all the important characters in each frame. This methodology allowed us to compare "computer vision" with "neuronal vision"--footprints associated with each character present in the activity of a subset of neurons--and identify the brain regions that contributed to this decoding process. We then tested the DL models during a recognition memory task following movie viewing where subjects were asked to recognize clip segments from the presented episode. DL model activations were not only modulated by the presence of the corresponding characters but also by participants subjective memory of whether they had seen the clip segment, and by the associative strengths of the characters in the narrative plot. The described approach can offer novel ways to probe the representation of concepts in time-evolving dynamic behavioral tasks. Further, the results suggest that the information required to robustly decode concepts is present in the population activity of only tens of neurons even in brain regions beyond MTL.

neuroscience↗

bioRxiv2021

Stochastic Consolidation of Lifelong Memory

Humans have the remarkable ability to continually store new memories, while maintaining old memories for a lifetime. How the brain avoids catastrophic forgetting of memories due to interference between encoded memories is an open problem in computational neuroscience. Here we present a model for continual learning in a recurrent neural network combining Hebbian learning, synaptic decay and a novel memory consolidation mechanism. Memories undergo stochastic rehearsals with rates proportional to the memorys basin of attraction, causing self-amplified consolidation, giving rise to memory lifetimes that extend much longer than synaptic decay time, and capacity proportional to a power of the number of neurons. Perturbations to the circuit model cause temporally-graded retrograde and anterograde deficits, mimicking observed memory impairments following neurological trauma.

neuroscience↗

bioRxiv2021

Cognitive boundary signals in the human medial temporal lobe shape episodic memory representation

While experience unfolds continuously, memories are organized as a set of discrete events that bind together the "where", "when", and "what" of episodic memory. This segmentation of continuous experience is thought to be facilitated by the detection of salient environmental or cognitive events. However, the underlying neural mechanisms and how such segmentation shapes episodic memory representations remain unclear. We recorded from single neurons in the human medial temporal lobe while subjects watched videos with different types of embedded boundaries and were subsequently evaluated for memories of the video contents. Here we show neurons that signal the presence of cognitive boundaries between subevents from the same episode and neurons that detect the abstract separation between different episodes. The firing rate and spike timing of these boundary-responsive neurons were predictive of later memory retrieval accuracy. At the population level, abrupt neural state changes following boundaries predicted enhanced memory strength but impaired order memory, capturing the behavioral tradeoff subjects exhibited when recalling episodic content versus temporal order. Successful retrieval was associated with reinstatement of the neural state present following boundaries, indicating that boundaries structure memory search. These findings reveal a neuronal substrate for detecting cognitive boundaries and show that cognitive boundary signals facilitate the mnemonic organization of continuous experience as a set of discrete episodic events.

neuroscience↗