Hedrich, N. L. | Search bioRxiv

bioRxiv2025

Entorhinal cortex signals dimensions of past experience that can be generalised in a novel environment

No two situations are identical. They can be similar in some aspects but different in others. This poses a key challenge when attempting to generalise our experience from one situation to another. How do we distinguish the aspects that transfer across situations from those that do not? One hypothesis is that the entorhinal cortex (EC) meets this challenge by forming factorised representations that allow for increased neural similarity between events that share generalisable features. We tested this hypothesis using functional magnetic resonance imaging. Female and male participants (n=40) were trained to report behavioural sequences based on an underlying graph structure. Participants then made decisions in a new environment where some, but not all graph transitions from the previous structure could be generalised. Behavioural results showed that participants distinguished the generalisable transition information. Accuracy was significantly higher in blocks where sequence transitions were shared across environments, than those in which transitions differed. This boost in accuracy was especially pronounced during early exposure to the novel environment. Throughout this early phase, neural patterns in EC showed a corresponding differentiation of the generalisable aspects. Neural patterns representing starting locations in familiar and novel environments were significantly more similar in EC on trials where sequences could be generalised from prior experience, compared to trials with new sequential transitions. This signalling was associated with improved performance when prior sequence knowledge could be reused. Our results suggest that during early exposure to novel environments, EC may signal dimensions of past experience that can be generalised. Significance StatementGeneralisation is the process of using our experience to solve new challenges. A central problem we face when attempting to generalise is determining which aspects of a new situation are similar to what we have experienced before and which are different. This allows us to generalise our knowledge selectively, transferring insights to the aspects that are similar and relevant in novel scenarios, while avoiding over-generalisation to the aspects that differ. The results from this experiment suggest that the entorhinal cortex may distinguish these aspects during early exposure to new environments, supporting complex generalisation behaviour.

neuroscience↗

bioRxiv2024

An inductive bias for slowly changing features in human reinforcement learning

Identifying goal-relevant features in novel environments is a central challenge for efficient behaviour. We asked whether humans address this challenge by relying on prior knowledge about common properties of reward-predicting features. One such property is the rate of change of features, given that behaviourally relevant processes tend to change on a slower timescale than noise. Hence, we asked whether humans are biased to learn more when task-relevant features are slow rather than fast. To test this idea, 295 human participants were asked to learn the rewards of two-dimensional bandits when either a slowly or quickly changing feature of the bandit predicted reward. Across two experiments and one preregistered replication, participants accrued more reward when a bandits relevant feature changed slowly, and its irrelevant feature quickly, as compared to the opposite. We did not find a difference in the ability to generalise to unseen feature values between conditions. Testing how feature speed could affect learning with a set of four function approximation Kalman filter models revealed that participants had a higher learning rate for the slow feature, and adjusted their learning to both the relevance and the speed of feature changes. The larger the improvement in participants performance for slow compared to fast bandits, the more strongly they adjusted their learning rates. These results provide evidence that human reinforcement learning favours slower features, suggesting a bias in how humans approach reward learning. Author summaryLearning experiments in the laboratory are often assumed to exist in a vacuum, where participants solve a given task independently of how they learn in more natural circumstances. But humans and other animals are in fact well known to "meta learn", i.e. to leverage generalisable assumptions about how to learn from other experiences. Taking inspiration from a well-known machine learning technique known as slow feature analysis, we investigated one specific instance of such an assumption in learning: the possibility that humans tend to focus on slowly rather than quickly changing features when learning about rewards. To test this, we developed a task where participants had to learn the value of stimuli composed of two features. Participants indeed learned better from a slowly rather than quickly changing feature that predicted reward. Computational modelling of participant behaviour indicated that participants had a higher learning rate for slowly changing features from the outset. Hence, our results support the idea that human reinforcement learning reflects a priori assumptions about the reward structure in natural environments.

neuroscience↗