Biology subjects

Izquierdo, A.

Publications and source records attributed to Izquierdo, A..

16 recordsLinked to original sources

bioRxiv2025

Sex differences in task engagement and lapse rate during reward learning plateaus

Our understanding of sex differences in reward learning has been limited due to the predominant study of males. Here, we evaluated sex differences in flexible learning in two domains: the learning of stimulus-and action-based associations and their reversals. During action-based learning, rats selected between two identical visual stimuli presented on a touchscreen, where the spatial location predicted a higher probability of reward. For stimulus-based learning, rats chose between two distinct visual stimuli presented in pseudorandom spatial locations, one of which was associated with a higher probability of reward. Reversal phases involved switching reward contingency between the two actions or stimuli. We found that females did not differ in their discrimination or reversal learning accuracy compared to males, yet they collected fewer rewards and were more likely to omit trials than males in both domains. To gain a detailed understanding of differences across conditions, we modeled animals trial-by-trial choices using reinforcement learning (RL) models and examined their steady-state behavior to capture transitions between distinct behavioral states. Although the estimated parameters of the best-fitting RL model revealed some sex differences, the model that incorporated transitions between different behavioral states provided a better overall fit to the data. This model also revealed that across all reversal phases, females exhibited a higher transition-specific lapse rate than males, indicating greater task disengagement once there was no need for further learning. Together, our fine-grained analysis of behavior adds to a growing literature on sex differences in flexible reward learning.

animal behavior and cognition↗

bioRxiv2025

A progressive ratio task with costly resets reveals adaptive effort-delay tradeoffs

The Progressive Ratio (PR) schedule is a popular test for measuring the motivational value of a reinforcer, in which subjects must exert an increasing amount of work to obtain each successive reward. Despite its popularity, the PR task hinges on a low-dimensional behavioral readout--breakpoint, or the maximum work requirement subjects are willing to complete before abandoning the task. Here, we show that with a simple modification, the PR task can be transformed into an optimization problem reminiscent of the patch-leaving foraging scenario, which has been analyzed extensively by behavioral ecologists, psychologists, and neuroscientists. In the Progressive Ratio with Reset (PRR) task, rats perform the PR task on one lever, but can press a second lever to reset the current ratio requirement back to its lowest value at the cost of enduring a reset delay, during which both levers are retracted. Rats used the reset lever adaptively on the PRR task, and their ratio reset decisions were sensitive to the cost of the reset delay. We derived an approach for computing the optimal bout length--the number of rewards to earn before pressing the reset lever that produces the greatest long-term rate of reward--and found that rats flexibly changed their behavior to approximate the optimal strategy. However, rats showed a systematic bias for bout lengths that exceeded the optimal length, an effect reminiscent of "overharvesting" in patch-leaving tasks. The PRR task thus represents a novel means of testing whether and how rats adapt their behavior to the cost-benefit structure of the environment in a way that connects deeply to the broader literature on associative learning and optimal foraging theory.

animal behavior and cognition↗

bioRxiv2024

Neural coding of choice and outcome are modulated by uncertainty in orbitofrontal but not secondary motor cortex

Orbitofrontal cortex (OFC) and secondary motor cortex (M2) are both implicated in flexible reward learning but the conditions that differentially recruit these regions are not fully understood. We imaged calcium activity from single neurons in OFC or M2 during de novo learning of increasingly uncertain reward probability schedules. Predictions of choice were decoded from M2 neurons with high accuracy under all certainty conditions, but were more accurately decoded from OFC neurons under greater uncertainty. The number of choice- and reward-selective neurons was significantly higher in M2 than in OFC across schedules, but these proportions increased across levels of uncertainty only in OFC. Decoding accuracy of choice and outcome was predicted by behavioral strategies Win-Stay and Lose-Shift in OFC, but not M2. We also tested causal involvement of these regions with chemogenetic perturbation as rats learned increasing and then decreasing uncertainty schedules. Whereas inhibition of OFC neurons attenuated learning across all schedules, M2 neurons were found to support learning in the most certain reward schedule. Thus, OFC neurons preferentially encode choices and outcomes that foster a greater reliance on adaptive strategies under uncertainty. This reveals a novel functional heterogeneity within frontal cortex in support of flexible learning.

neuroscience↗

bioRxiv2024

A common stay-on-goal mechanism in anterior cingulate cortex for information and effort choices

Humans and non-humans alike often make choices to gain information, even when the information cannot be used to change the outcome. Prior research has shown the anterior cingulate cortex (ACC) is important for evaluating options involving reward-predictive information. Here we studied the role of ACC in information choices using optical inhibition to evaluate the contribution of this region during specific epochs of decision making. Rats could choose between an uninformative option followed by a cue that predicted reward 50% of the time vs. a fully informative option that signaled outcomes with certainty, but was rewarded only 20% of the time. Reward seeking during the informative S+ cue decreased following ACC inhibition, indicating a causal contribution of this region in supporting reward expectation to a cue signaling reward with certainty. Separately in a positive control experiment and in support of a known role for this region in sustaining high-effort behavior for preferred rewards, we observed reduced lever presses and lower breakpoints in effort choices following ACC inhibition. The lack of changes in reward latencies in both types of decisions indicate the motivational value of rewards remained intact, revealing instead a common role for ACC in maintaining persistence toward certain and valuable rewards. Graphical Summary O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=59 SRC="FIGDIR/small/619920v2_ufig1.gif" ALT="Figure 1"> View larger version (11K): org.highwire.dtl.DTLVardef@6fa3b2org.highwire.dtl.DTLVardef@1642fa8org.highwire.dtl.DTLVardef@74d4d9org.highwire.dtl.DTLVardef@421297_HPS_FORMAT_FIGEXP M_FIG C_FIG Significance StatementWe often make choices to gain information, even when the information cannot be used to change the outcome. Here we investigated the precise timing of the role of the anterior cingulate cortex (ACC) in decisions that involve seeking certain versus uncertain rewards. By optically inhibiting ACC neurons, we demonstrate that this region is crucial for maintaining persistence toward rewards signaled with certainty, without altering the motivational value of the reward itself. In a positive control experiment, we also confirm that ACC is important in effort-based choice. The findings reveal a common role for ACC in maintaining persistence toward certain and valuable rewards, necessary for making optimal decisions. These results have implications for understanding psychiatric disorders involving maladaptive reward-seeking behavior.

neuroscience↗

bioRxiv2023

A special role for Anterior Cingulate Cortex, but not Orbitofrontal Cortex or Basolateral Amygdala, in choices involving information

Subjects often are willing to pay a cost for information. In a procedure that promotes paradoxical choices, animals choose between a richer option followed by a cue that is rewarded 50% of the time (No-info) vs a leaner option followed by one of two cues that signal certain outcomes: one always rewarded (100%), and the other never rewarded, 0% (Info). Since decisions involve comparing the subjective value of options after integrating all their features, preference for information may rely on cortico-amygdalar circuitry. To test this, male and female rats were prepared with bilateral inhibitory DREADDs in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), basolateral amygdala (BLA), or null virus (control). We inhibited these regions after stable preference was acquired. We found that inhibition of ACC destabilized choice preference in female rats without affecting latency to choose or response rate to cues. A logistic regression fit revealed that the previous choice strongly predicted preference in control animals, but not in female rats following ACC inhibition. The results reveal a causal, sex-dependent role for ACC in decisions involving information.

neuroscience↗

bioRxiv2023

Sex-dependent contributions of ventrolateral orbitofrontal cortex and basolateral amygdala to learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment. Significance StatementInflexible learning is a feature of several neuropsychiatric disorders. We investigated how the ventrolateral orbitofrontal cortex (vlOFC) and basolateral amygdala (BLA) are involved in learning of stimuli or actions under reinforcement uncertainty. Following chemogenetic inhibition of these regions in both male and females, we measured learning and adjustments to deterministic and probabilistic reversals. For action learning, BLA, but not vlOFC, is needed for probabilistic reversal learning. However, BLA is not necessary for initial probabilistic learning or retention, indicating a critical role for learning of unexpected changes. For stimulus learning, vlOFC, but not BLA, is required for adjustments to reversals, particularly in females. These findings provide insight into the complementary cortico-amygdalar substrates of learning under different forms of uncertainty.

neuroscience↗

bioRxiv2023

A novel hypothalamic-midbrain circuit for model-based learning

Behavior is often dichotomized into model-free and model-based systems 1, 2. Model-free behavior prioritizes associations that have high value, regardless of the specific consequence or circumstance. In contrast, model-based behavior involves considering all possible outcomes to produce behavior that best fits the current circumstance. We typically exhibit a mixture of these behaviors so we can trade-off efficiency and flexibility. However, substance use disorder shifts behavior more strongly towards model-free systems, which produces a difficulty abstaining from drug-seeking due to an inability to withhold making the model-free high-value response 3-10. The lateral hypothalamus (LH) is implicated in substance use disorder 11-17 and we have demonstrated that this region is critical to Pavlovian cue-reward learning 18, 19. However, it is unknown whether learning occurring in LH is model-free or model-based, where the necessary teaching signal comes from to facilitate learning in LH, and whether this is relevant for learning deficits that drive substance use disorder. Here, we reveal that learning occurring in the LH is model-based. Further, we confirm the existence of an understudied projection extending from dopamine neurons in the ventral tegmental area (VTA) to the LH and demonstrate that this input underlies model-based learning in LH. Finally, we examine the impact of methamphetamine self-administration on LH-dependent model-based processes. These experiments reveal that a history of methamphetamine administration enhances the model-based control that Pavlovian cues have over decision-making, which was accompanied by a bidirectional strengthening of the LH to VTA circuit. Together, this work reveals a novel bidirectional circuit that underlies model-based learning and is relevant to the behavioral and cognitive changes that arise with substance use disorders. This circuit represents a new addition to models of addiction, which focus on instrumental components of drug addiction and increases in model-free habits after drug exposure 3-10.

neuroscience↗

bioRxiv2022

Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of entropy-based metrics in studying adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

neuroscience↗

bioRxiv2022

A hardware system for real-time decoding of in-vivo calcium imaging data

Epifluorescence miniature microscopes ("miniscopes") are widely used for in vivo calcium imaging of neural population activity. Imaging data is usually collected while subjects are engaged in a task and stored for later offline analysis, but emerging techniques for online imaging offer potential for novel real-time experiments in which closed-loop interventions (such as neurostimulation or sensory feedback) are triggered at short latencies in response to neural population activity. Here we introduce DeCalciOn, a plug-and-play hardware device for online population decoding of in vivo calcium signals that can trigger closed-loop feedback at millisecond latencies, and is compatible with miniscopes that use the UCLA Data Acquisition (DAQ) interface. In performance tests, the position of rats (n=13) on a linear track was decoded in real time from hippocampal CA1 population activity by 24 linear classifiers. DeCalciOn required <2.5 ms after each end-of-frame to decode up to 1,024 calcium traces and trigger TTL control outputs. Decoding was most efficient using a contour-free method of extracting traces from ROIs that were unaligned with neurons in the image, but contour-based extraction from neuronal ROIs is also supported. DeCalciOn is an easy-to-use system for real-time decoding of calcium fluorescence that enables closed-loop feedback experiments in behaving animals.

neuroscience↗

bioRxiv2021

Theta oscillations in frontal cortex differentially modulate accuracy and speed in flexible stimulus- and action-based learning

Flexible reward learning relies on frontal cortex, with substantial evidence indicating that anterior cingulate cortex (ACC) and orbitofrontal cortex (OFC) subregions play important roles. Recent studies in both rat and macaque suggest theta oscillations (5-10 Hz) may be a spectral signature that coordinates this learning. However, network-level interactions between ACC and OFC in flexible learning remain unclear. We investigated the learning of stimulus-reward associations using a combination of simultaneous in-vivo electrophysiology in dorsal ACC and ventral OFC, partnered with bilateral inhibitory DREADDs in ACC. In freely-behaving male and female rats and using a within-subject design, we examined accuracy and speed of response across distinct and precisely-defined trial epochs during initial visual discrimination learning and subsequent reversal of stimulus-reward contingencies. Following ACC inhibition there was a propensity for random responding in early reversal learning, with correct vs. incorrect trials distinguished only from OFC, not ACC, theta power differences in the reversal phase. ACC inhibition also hastened incorrect choices during reversal. This same pattern of change in accuracy and speed was not observed in viral control animals. Thus, characteristics of impaired reversal learning following ACC inhibition are poor deliberation and weak theta signaling of accuracy in this region. The present results also point to OFC theta oscillations as a prominent feature of reversal learning, unperturbed by ACC inhibition.

neuroscience↗

bioRxiv2021

Comprehensive evolutionary analysis of complete Epstein Barr virus genomes from Argentina and other geographies

Epstein Barr virus sequence variability has been deeply studied throughout the past years in isolates from various geographic regions and consequent geographic variation at both genetic and genomic levels has been described. However, isolates from South America have been underrepresented in these studies. Here, we sequenced 15 complete EBV genomes that we analyzed, by means of a custom-built bioinformatic pipeline, together with publicly available raw NGS data for 199 EBV isolates from other parts of the globe. Phylogenetic relations of the genomes, geographic structure and variability of the data set, and evolution rates for the whole genome and each gene were assessed. The present study contributes to overcome the scarcity of EBV complete genomes from South America, and hence, achieves the most comprehensive geography-related variability study by determining the actual contribution of each EBV gene to the geographic segregation of the entire genomes. Moreover, to the best of our knowledge, we established for the first time the evolution rate for the entire EBV genome, on a host-virus codivergence-independent assumption, and statistically demonstrate that evolution rates, on a gene-by-gene basis, are related to the encoded protein function. Considering evolution of dsDNA viruses with a codivergence-independent approach, may lay the basis for future research on EBV evolution. Additionally, this work also expands the sampling-time lapse of available complete genomes derived from different EBV-related conditions, a matter that until today, prevents for detailed phylogeographic analysis.

evolutionary biology↗

bioRxiv2020

The Orbitofrontal Cortex in Temporal Cognition

One of the most important factors in decision making is estimating the value of available options. Subregions of the prefrontal cortex, including the orbitofrontal cortex (OFC), have been deemed essential for this process. Value computations require a complex integration across numerous dimensions, including, reward magnitude, effort, internal state, and time. The importance of the temporal dimension is well-illustrated by temporal discounting tasks, in which subjects select between smaller-sooner versus larger-later rewards. The specific role of OFC in telling time and integrating temporal information into decision making remains unclear. Based on the current literature, in this review we reevaluate current theories of OFC function, accounting for the influence of time. Incorporating temporal information into value estimation and decision making requires distinct, yet interrelated, forms of temporal information including the ability to tell time, represent time, create temporal expectations, and the ability to use this information for optimal decision making in a wide range of tasks, including temporal discounting and wagering. We use the term temporal cognition to refer to the integrated use of these different aspects of temporal information. We suggest that the OFC may be a critical site for the integration of reward magnitude and delay, and thus important for temporal cognition.

neuroscience↗

bioRxiv2020

Unique features of stimulus-based probabilistic reversal learning

Reversal learning paradigms are widely-used assays of behavioral flexibility with their probabilistic versions being more amenable to studying integration of reward outcomes over time. Prior research suggests differences between initial and reversal learning, including higher learning rates, a greater need for inhibitory control, and more perseveration after reversals. However, it is not well-understood what aspects of stimulus-based reversal learning are unique to reversals, and whether and how observed differences depend on reward probability. Here, we used a visual probabilistic discrimination and reversal learning paradigm where male and female rats selected between a pair of stimuli associated with different reward probabilities. We compared accuracy, rewards collected, omissions, latencies, win-stay/lose-shift strategies, and indices of perseveration across two different reward probability schedules. We found that discrimination and reversal learning are behaviorally more unique than similar: latencies to select the better option, win-stay strategies, and perseveration were of a different pattern in reversal learning compared to discrimination learning. Additionally, fit of choice behavior using reinforcement learning models revealed a lower sensitivity to the difference in subjective reward values (greater exploration) and higher learning rates for the reversal phase. Interestingly, a consistent reward probability group difference emerged with a richer environment associated with longer reward collection latencies than a leaner environment. We also replicated previous reports on sex differences in reversal learning. Future studies should systematically compare the neural correlates of fine-grained behavioral measures to reveal possible dissociations in how the circuitry is recruited in each phase.

animal behavior and cognition↗

bioRxiv2020

Sex-dependent effects of chronic intermittent voluntary alcohol consumption on attentional, not motivational, measures during probabilistic learning and reversal

BackgroundForced alcohol (ethanol, EtOH) exposure has been shown to cause significant impairments on reversal learning, a widely-used assay of cognitive flexibility, specifically on fully-predictive, deterministic versions of this task. However, previous studies have not adequately considered voluntary EtOH consumption and sex effects on probabilistic reversal learning. The present study aimed to fill this gap in the literature. MethodsMale and female Long-Evans rats underwent either 10 weeks of voluntary intermittent 20% EtOH access or water only (H2O) access. Rats were then pretrained to initiate trials and learn stimulus-reward associations via touchscreen response, and subsequently required to select between two visual stimuli, rewarded with probability 0.70 or 0.30. In the final phase, reinforcement contingencies were reversed. ResultsWe found significant sex differences on several EtOH-drinking variables, with females reaching a higher maximum EtOH consumption, exhibiting more high-drinking days, and escalating their EtOH at a quicker rate compared to males. During early abstinence, EtOH drinkers made more initiation omissions and were slower to initiate trials than H2O drinking controls, particularly during pretraining. EtOH drinking females were most affected. A similar pattern in trial initiations was also observed in discrimination, but not in reversal learning. EtOH drinking rats were unaffected in their reward collection and stimulus response times, indicating intact motivation and motor responding. Although there were sex differences in discrimination and reversal phases, performance improved over time. We also observed sex-independent drinking group differences in win-stay and lose-shift strategies specific to the reversal phase. ConclusionsFemales exhibit increased vulnerability to EtOH effects in early learning: there were sex-dependent EtOH effects on attentional measures during pretraining and discrimination phases. We also found sexindependent EtOH effects on exploration strategies during reversal. Future studies should aim to uncover the neural mechanisms for changes in attention and exploration in both acute and prolonged EtOH withdrawal.

pharmacology and toxicology↗

bioRxiv2019

Anterior cingulate cortex activity regulates effort-based decision making

The anterior cingulate cortex (ACC) is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC--via either chemogenetic inhibition or excitation--reduced lever pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we utilized in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared to no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Taken together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options. Significance StatementThe anterior cingulate cortex (ACC) is implicated in effort-based decision making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared to the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.

neuroscience↗

bioRxiv2019

Dissociable roles for Anterior Cingulate Cortex and Basolateral Amygdala in Decision Confidence and Learning under Uncertainty

It has been suggested the subjective sense of certainty, or confidence, in ambiguous sensory cues can alter the interpretation of reward feedback and facilitate learning. We trained rats to report the orientation of ambiguous visual stimuli according to a spatial stimulus-response rule. Following choice, rats could wait a self-timed delay for reward or initiate a new trial. Waiting times increased with discrimination accuracy, demonstrating that this measure could be used as a proxy for confidence. Chemogenetic silencing of BLA shortened waiting times overall whereas ACC inhibition rendered waiting times insensitive to confidence-modulating attributes of visual stimuli, suggesting contribution of ACC but not BLA to confidence computations. Subsequent reversal learning was enhanced by confidence. Both ACC and BLA inhibition blocked this enhancement but via differential modulation of learning strategies and consistency in using learned rules. Altogether, we demonstrate dissociable roles for ACC and BLA in transmitting confidence and learning under uncertainty.

neuroscience↗