Biology subjects

Nili Ahmadabadi, M.

Publications and source records attributed to Nili Ahmadabadi, M..

6 recordsLinked to original sources

bioRxiv2025

Deformed Probability Estimation in Goal-Directed reinforcement learning model explains anxious-depression dimensions of psychiatric disorders

Psychiatric disorders are complex, multi-dimensional pathologies rooted in diverse cognitive processes. Computational psychiatry aims to reveal distortions in these processes through behavior modeling, providing a deeper understanding of psychiatric disorders. Previous studies, using Daws two-stage task, had linked the imbalance between habitual/model-free and goal-directed/model-based behaviors to disorders with compulsive behaviors and intrusive thoughts. The model-based component relies on the estimation of environmental probabilities. Therefore, we added a well-known deformation in subjective probability estimation to the model and this improved the model fitting. More importantly, the fitted deformation explains some variance of the anxious-depression dimension of psychiatric symptoms. The deformation parameter is aligned with the description-experience gap in decision-making literature. Our results point to subjective possibly distortion as the probable underlying cognitive process of anxiety, apathy, and depression. This study also shows that the inclusion of cognitive biases in modeling can extract the hidden aspects of behavior possibility linked to disorders. Our approach enhances the precision of computational psychiatry and provides deeper insights into the cognitive processes underlying psychiatric symptoms, paving the way for more effective, personalized therapeutic strategies. Significance StatementThis study builds on a previously used model and Data that indicated a correlation between Model-Based preference and certain psychiatric disorders. By adding distortion to the probability estimation, we identified a new parameter correlated with depression and anxiety. The augmented model demonstrates an improved fit to behavior and aligns with Gillans previous findings. Our approach enhances the precision of computational psychiatry and provides deeper insights into the cognitive processes underlying psychiatric symptoms, paving the way for more effective, personalized therapeutic strategies.

neuroscience↗

bioRxiv2021

Optimal Reinforcement Learning with Asymmetric Updating in Volatile Environments: a Simulation Study

AO_SCPLOWBSTRACTC_SCPLOWThe ability to predict the future is essential for decision-making and interaction with the environment to avoid punishment and gain reward. Reinforcement learning algorithms provide a normative way for interactive learning, especially in volatile environments. The optimal strategy for the classic reinforcement learning model is to increase the learning rate as volatility increases. Inspired by optimistic bias in humans, an alternative reinforcement learning model has been developed by adding a punishment learning rate to the classic reinforcement learning model. In this study, we aim to 1) compare the performance of these two models in interaction with different environments, and 2) find optimal parameters for the models. Our simulations indicate that having two different learning rates for rewards and punishments increases performance in a volatile environment. Investigation of the optimal parameters shows that in almost all environments, having a higher reward learning rate compared to the punishment learning rate is beneficial for achieving higher performance which in this case is the accumulation of more rewards. Our results suggest that to achieve high performance, we need a shorter memory window for recent rewards and a longer memory window for punishments. This is consistent with optimistic bias in human behavior.

neuroscience↗

bioRxiv2020

Implicit counterfactual effect in partial feedback reinforcement learning: behavioral and modeling approach

Context by distorting values of options with respect to the distribution of available alternatives, remarkably affects learning behavior. Providing an explicit counterfactual component, outcome of unchosen option alongside with the chosen one (Complete feedback), would increase the contextual effect by inducing comparison-based strategy during learning. But It is not clear in the conditions where the context consists only of the juxtaposition of a series of options, and there is no such explicit counterfactual component (Partial feedback), whether and how the relativity will be emerged. Here for investigating whether and how implicit and explicit counterfactual components can affect reinforcement learning, we used two Partial and Complete feedback paradigms, in which options were associated with some reward distributions. Our modeling analysis illustrates that the model which uses the outcome of chosen option for updating values of both chosen and unchosen options, which is in line with diffusive function of dopamine on the striatum, can better account for the behavioral data. We also observed that size of this bias depends on the involved systems in the brain, such that this effect is larger in the transfer phase where subcortical systems are more involved, and is smaller in the deliberative value estimation phase where cortical system is more needed. Furthermore, our data shows that contextual effect is not only limited to probabilistic reward but also it extends to reward with amplitude. These results show that by extending counterfactual concept, we can better account for why there is contextual effect in a condition where there is no extra information of unchosen outcome.

animal behavior and cognition↗

bioRxiv2019

Higher Meta-cognitive Ability Predicts Less Reliance on Over Confident Habitual Learning System

Many studies on human and animals have provided evidence for the contribution of goal-directed and habitual valuation systems in learning and decision-making. These two systems can be modeled using model-based (MB) and model-free (MF) algorithms in Reinforcement Learning (RL) framework. Here, we study the link between the contribution of these two learning systems to behavior and meta-cognitive capabilities. Using computational modeling we showed that in a highly variable environment, where both learning strategies have chance level performances, model-free learning predicts higher confidence in decisions compared to model-based strategy. Our experimental results showed that the subjects meta-cognitive ability is negatively correlated with the contribution of model-free system to their behavior while having no correlation with the contribution of model-based system. Over-confidence of the model-free system justifies this counter-intuitive result. This is a new explanation for individual difference in learning style.

neuroscience↗

bioRxiv2018

Uncovering Network Architecture Using an Exact Statistical Input-Output Relation of a Neuron Model

Using observed neuronal activity, we try to unveil hidden microcircuits. A key requirement is the knowledge of statistical input-output relation of single neurons in vivo. We use a recent exact solution of spike-timing for leaky integrate-and-fire neurons under noisy inputs balanced near threshold, and construct a framework that links synaptic type/strength, and spiking nonlinearity, with statistics of neuronal activity. The framework explains structured higher-order interactions of neurons receiving common inputs under different architectures. Comparing models prediction with an empirical dataset of monkey V1 neurons, we find that excitatory inputs to pairs explain the observed sparse activity characterized by negative triple-wise interactions, ruling out the intuitive shared inhibition. We show that the strong interactions are in general the signature of excitatory rather than inhibitory inputs whenever spontaneous activity is low. Finally, we present a guide map that can be used to reveal the hidden motifs underlying observed interactions found in empirical data.

neuroscience↗

bioRxiv2018

Better than maximum likelihood estimation of model-based and model-free learning style

Multiple decision making systems work together to shape the final choices in human behavior. Habitual and goal-directed systems are the two most important systems that are studied in the reinforcement learning (RL) literature by model-free and model-based learning methods. Human behavior resembles the weighted combination of these systems and such a combination is modeled by weighted summation of actions value from the model based and model free systems. Extraction of this weighted parameter, which is important for many applications and computational modeling, has been mostly based on the maximum likelihood or maximum a posteriori methods. We show these methods bring many challenges and their respective extracted values are less reliable especially in the proximity of extremes values. We propose that using a free format learning method (k-nearest neighbor) which uses more information besides the fitted values e.g. global information like stay probability instead of trial by trial information can ameliorate the estimation error. The proposed method is examined by simulation and results show the advantage of the proposed method. In addition, investigation of the human behavior data from previous researchers proved the proposed method to result in more statistically robust results in predicting other behavioral indices such as the number of gaze directions toward each target. In brief, the proposed method increases the reliability of the estimated parameters and enhances the applicability of reinforcement learning paradigms in clinical trials.

neuroscience↗