Biology subjects

Waernberg, E.

Publications and source records attributed to Waernberg, E..

3 recordsLinked to original sources

bioRxiv2022

Vector-valued dopamine improves learning of continuous outputs in the striatum

It is well established that midbrain dopaminergic neurons support reinforcement learning (RL) in the basal ganglia by transmitting a reward prediction error (RPE) to the striatum. In particular, different computational models and experiments have shown that a striatumwide RPE signal can support RL over a small discrete set of actions (e.g. no/no-go, choose left/right). However, there is accumulating evidence that the basal ganglia functions not as a selector between predefined actions, but rather as a dynamical system with graded, continuous outputs. To reconcile this view with RL, there is a need to explain how dopamine could support learning of dynamic outputs, rather than discrete action values. Inspired by the recent observations that besides RPE, the firing rates of midbrain dopaminergic neurons correlate with motor and cognitive variables, we propose a model in which dopamine signal in the striatum carries a vector-valued error feedback signal (a loss gradient) instead of a homogeneous scalar error (a loss). Using a recurrent network model of the basal ganglia, we show that such a vector-valued feedback signal results in an increased capacity to learn a multidimensional series of real-valued outputs. The corticostriatal plasticity rule we employed is based on Random Feedback Learning Online learning and is a fully local, "three-factor" product of the presynaptic firing rate, a post-synaptic factor and the unique dopamine concentration perceived by each striatal neuron. Crucially, we demonstrate that under this plasticity rule, the improvement in learning does not require precise nigrostriatal synapses, but is compatible with random placement of varicosities and diffuse volume transmission of dopamine.

neuroscience↗

bioRxiv2020

Complete representation of action space and value in all striatal pathways.

The dorsal striatum plays a central role in motor and decision programs, such as the selection and execution of particular actions and the evaluation of their outcomes. A standard model has emerged where distinct output pathways encode separate motor-action signals, including selection-evaluation division in the matrix versus patch compartments. We used large-scale cell-type specific calcium imaging during motor and decision behaviors to determine and contrast the activity of individual striatal projection neurons (SPNs) belonging to one of the three major output pathways in the dorsomedial striatum - patch Oprm1+ SPNs versus the D1+ direct and A2A+ indirect pathway. We found that Oprm1+ SPNs were tuned to a number of different behavioral categories, such as to different movements, or to discrete actions and decisions in a two-choice task, and these complex representations were found to the same extent in all three striatal output pathways. The sharp tuning of individual SPNs was highly stereotyped over time while performing a specific task, but the tuning profile remapped between different behavioral contexts. In addition to action representations, SPNs showed pathway-independent representation of decision-variables such as the trial strategy and the action value. We propose that all three major output pathways in the dorsomedial striatum share a similarly complete representation of the entire action space, including task- and phase-specific signals of action value and choice.

neuroscience↗

bioRxiv2017

Low Dimensional Activity in Spiking Neuronal Networks

Several recent studies have shown that neural activity in vivo tends to be constrained to a low-dimensional manifold. Such activity does not arise in simulated neural networks with homogeneous connectivity and it has been suggested that it is indicative of some other connectivity pattern in neuronal networks. Surprisingly, the structure of the intrinsic manifold of the network activity puts constraints on learning. For instance, animals find it difficult to perform tasks that may require a change in the intrinsic manifold. Here, we demonstrate that the Neural Engineering Framework (NEF) can be adapted to design a biologically plausible spiking neuronal network that exhibit low dimensional activity. Consistent with experimental observations, the resulting synaptic weight distribution is heavy-tailed (log-normal). In our model, a change in the intrinsic manifold of the network activity requires rewiring of the whole network, which may be either not possible or a very slow process. This observation provides an explanation of why learning is easier when it does not require the neural activity to leave its intrinsic manifold.\n\nSignificance statementA network in the brain consists of thousands of neurons. A priori, we expect that the network will have as many degrees of freedom as its number of neurons. Surprisingly, experimental evidence suggests that local brain activity is confined to a space spanned by 10 variables. Here, we describe an approach to construct spiking neuronal networks that exhibit low-dimensional activity and address the question: how the intrinsic dimensionality of the network activity restricts the learning as suggested by recent experiments? Specifically, we show that tasks that requires animals to change the network activity outside the intrinsic space would entail large changes in the neuronal connectivity, and therefore, animals are either slow or not able to acquire such tasks.

neuroscience↗