Biology subjects

Matias, S.

Publications and source records attributed to Matias, S..

6 recordsLinked to original sources

bioRxiv2026

The neural circuit implementation of distributional reinforcement learning

Distributional reinforcement learning (RL) algorithms outperform traditional RL by representing the full probability distribution of future outcomes rather than its mean. Neural signatures consistent with distributional coding have been observed in dopamine neurons and in striatal value representations. However, how distributional RL is implemented within the anatomical circuit organization of the basal ganglia remains unknown. Uncovering how representations of outcome distributions are organized across brain circuits has broad implications for both fundamental and clinical neuroscience, providing new insights into how the brain represents uncertainty and how these representations shape risk-sensitive learning and decision-making. Here, we investigated how representations of outcome distributions are organized across dopamine projections to the striatum by combining striatum-wide fiber photometry of dopamine axonal activity with Neuropixels recordings from projection-defined dopamine neurons. We found that distributional reward prediction error signals are preferentially encoded in lateral ventral striatum-projecting dopamine neurons, but that reward distribution information is shared between the ventral and the dorsal striatum. Additionally, an anatomical gradient of optimism in the midbrain supports the diversity of value predictions required for distributional RL. Our findings extend the actor-critic framework proposed for the basal ganglia by revealing distributional prediction errors in a subpopulation of mesolimbic dopamine neurons capable of training parallel striatal regions thought to implement different types of behavioral control.

neuroscience↗

bioRxiv2025

A hardwired neural circuit for temporal difference learning

The neurotransmitter dopamine plays a major role in learning by acting as a teaching signal to update the brains predictions about rewards. A leading theory proposes that this process is analogous to a reinforcement learning algorithm called temporal difference (TD) learning, and that dopamine acts as the error term within the TD algorithm (TD error). Although many studies have demonstrated similarities between dopamine activity and TD errors1-5, the mechanistic basis for dopaminergic TD learning remains unknown. Here, we combined large-scale neural recordings with patterned optogenetic stimulation to examine whether and how the key steps in TD learning are accomplished by the circuitry connecting dopamine neurons and their targets. Replacing natural rewards with optogenetic stimulation of dopamine axons in the nucleus accumbens (NAc) in a classical conditioning task gradually generated TD error-like activity patterns in dopamine neurons by specifically modifying the task-related activity of NAc neurons expressing the D1 dopamine receptor (D1 neurons). In turn, patterned optogenetic stimulation of NAc D1 neurons in naive animals drove dopamine neuron spiking according to the TD error of the stimulation pattern, indicating that TD computations are hardwired into this circuit. The transformation from D1 neurons to dopamine neurons could be described by a biphasic linear filter, with a rapid positive and delayed negative phase, that effectively computes a temporal difference. This finding suggests that the time horizon over which the TD algorithm operates--the temporal discount factor--is set by the balance of the positive and negative components of the linear filter, pointing to a circuit-level mechanism for temporal discounting. These results provide a new conceptual framework for understanding how the computations and parameters governing animal learning arise from neurobiological components.

neuroscience↗

bioRxiv2024

The role of prospective contingency in the control of behavior and dopamine signals during associative learning

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. Here we examined the dopamine activity in the ventral striatum - a signal implicated in associative learning - in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a novel causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate inter-trial-interval (ITI) state representation. Recurrent neural networks trained within a TD framework develop state representations like our best handcrafted model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

neuroscience↗

bioRxiv2024

Interpretable deep learning for deconvolutional analysis of neural signals

The widespread adoption of deep learning to build models that capture the dynamics of neural populations is typically based on "black-box" approaches that lack an interpretable link between neural activity and network parameters. Here, we propose to apply algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We characterize our method, referred to as deconvolutional unrolled neural learning (DUNL), and show its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. To exemplify use cases of our decomposition method, we uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons in an unbiased manner, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the heterogeneity of neural responses in the piriform cortex and in the striatum during unstructured, naturalistic experiments. Our work leverages the advances in interpretable deep learning to gain a mechanistic understanding of neural activity.

neuroscience↗

bioRxiv2024

An opponent striatal circuit for distributional reinforcement learning

Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards -- an approach known as distributional reinforcement learning (RL)1. The mesolimbic dopamine system is thought to underlie RL in the mammalian brain by updating a representation of mean value in the striatum2,3, but little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions4. To fill this gap, we used high-density probes (Neuropixels) to acutely record striatal activity from well-trained, water-restricted mice performing a classical conditioning task in which reward mean, reward variance, and stimulus identity were independently manipulated. In contrast to traditional RL accounts, we found robust evidence for abstract encoding of variance in the striatum. Remarkably, chronic ablation of dopamine inputs disorganized these distributional representations in the striatum without interfering with mean value coding. Two-photon calcium imaging and optogenetics revealed that the two major classes of striatal medium spiny neurons -- D1 and D2 MSNs -- contributed to this code by preferentially encoding the right and left tails of the reward distribution, respectively. We synthesize these findings into a new model of the striatum and mesolimbic dopamine that harnesses the opponency between D1 and D2 MSNs5-15 to reap the computational benefits of distributional RL.

neuroscience↗

bioRxiv2023

Genetic Screening Of Halothane Gene On Selected Philippine Native Pig Herds

Establishment of nucleus herds (NHs) of Native Pigs (NPs) at various R&D stations in the Philippines is currently being undertaken for food security and genetic conservation. Marker-assisted selection (MAS) is being utilized to identify individuals carrying favorable alleles of genes associated with production traits and screen-out genetic defects (GD) for breeding purposes. Porcine Stress Syndrome (PSS) caused by a mutation in Halothane (HAL) gene is a GD frequently found in commercial breeds that when expressed, causes pale, soft, exudative (PSE) meat. PSE is inferior quality meat undesirable in the market causing economic loss to the swine industry. Thus, this study was conducted to screen HAL gene using mutagenically separated-polymerase chain reaction (MS-PCR) in selected NP herds and assessed its repeatability in local breeds. Results showed that out of 577 screened individuals, 543 (94.11%) were normal (NN), 0 (0%) were homozygous mutant (nn) and 34 (5.89%) were heterozygous carriers (Nn). Therefore, the optimized PSS screening protocol using MSPCR is also applicable to local breed as described in the previous study. As such the availability of genetic test for PSS could be useful in improving the breeding selection and elimination of PSS mutant in the nucleus herd of Philippine Native Pig.

genetics↗