Santhosh, A. | Search bioRxiv

bioRxiv2026

Task-dependence of network-to-network variability in learning, performance, and dynamics of heterogeneous recurrent networks

Artificial recurrent networks are powerful models for studying neural dynamics and representations underlying complex cognitive tasks. However, the impact of neural-circuit heterogeneities on learning, dynamics, robustness, and generalization in these networks remains poorly understood. Here, we systematically investigated the impact of graded intrinsic heterogeneities in artificial recurrent networks trained on different cognitive tasks using reward- modulated Hebbian learning. Across networks trained with distinct hyperparameters and different levels of intrinsic heterogeneity, we observed pronounced network-to-network and task-to-task variability in training convergence, error dynamics during training, and task performance. These effects were strongly task dependent, with memory-dependent tasks exhibiting greater sensitivity to heterogeneity than memoryless tasks. We assessed these networks for robustness to multiple forms of graded post-training perturbations. Perturbations to intrinsic time constant distributions altered network dynamics, but had limited impact on final task accuracy in most cases. In contrast, perturbations to initial conditions, exploratory activity impulses, or task epoch durations strongly affected memory-dependent tasks. Among all perturbations, synaptic jitter was consistently the most detrimental, impairing performance across all tasks and heterogeneity levels. Importantly, despite such pronounced impact of heterogeneities, none of the metrics (spanning training, performance, dynamics, and robustness) varied monotonically with the level of training heterogeneity, instead showing additional dependencies on task demands, network configuration, and perturbation type. Finally, networks trained on a single task were able to perform structurally related untrained tasks, but failed on fundamentally distinct tasks. Strikingly, similar task performances emerged from divergent activity trajectories across networks and training conditions, together revealing pronounced functional degeneracy in network dynamics. Collectively, our findings establish that heterogeneous recurrent networks operate in a complex systems regime, where robust function emerges from non-unique, task-specific interactions among hyperparameters, dynamics, and heterogeneities. Our analyses emphasize the need for population- of-networks approaches that focus on interactions among multiple forms of neural heterogeneities in shaping learning and computation.

neuroscience↗

bioRxiv2025

Diversity in the impact of heterogeneities on recurrent networks performing a cognitive task

Background and motivationArtificial recurrent networks are widely used as models to study the complex dynamics underlying biological neural networks during execution of cognitive tasks. However, most studies assume individual units in the recurrent network to be homogeneous repeating units, whereas real neurons exhibit several forms of heterogeneities. In this study, we designed and employed a systematic framework for quantitative assessment of the impact of neural heterogeneities on recurrent networks that were trained to perform a cognitive task. MethodologyOur framework involved training of a population of recurrent networks, differing in terms of their hyperparameters, to perform a cognitive task in the presence of six graded levels of intrinsic heterogeneities. We tested the impact of heterogeneities on several performance metrics that encompassed training performance, task-execution dynamics, and resilience to different forms of post-training heterogeneities (also introduced at different levels). ResultsOur population-of-networks approach demonstrate that intrinsic heterogeneities impacted network performance and dynamics in diverse ways even if they were trained with the same training algorithm, convergence criteria, and task specifications. First, our analyses unveiled pronounced network-to-network variability in the dependence of training performance on the level of heterogeneity, in terms of the number of training trials required for learning and the error values associated with task performance. Second, the impact of training heterogeneities on network dynamics during task execution also manifested substantial variability across networks. Finally, our analyses revealed a prominent impact of different forms of post-training heterogeneities on performance errors and network dynamics. We observed progressive increases in errors as well as in trajectory deviations with graded increases in post-training heterogeneities. Importantly, we observed pronounced variability in how robustness to post-training heterogeneities depended on the level of training heterogeneities. Specifically, certain networks showed enhanced robustness to post-training heterogeneities when training heterogeneities were low, whereas others showed better robustness when training heterogeneities were high. ImplicationsThe striking nature of network-to-network variability observed in our analyses strongly advocates a complex systems viewpoint to study the impact of neural heterogeneities on circuit function. Within such a complex system framework, where several functionally specialized subsystems interact with each other in non-random ways to yield collective performance of the task, we argue that the emphasis should not be on heterogeneities in individual components of neural circuits. Instead, we emphasize the need to focus on the global structure of different forms and degrees of heterogeneities across different components and a systematic assessment of how they interact with each other towards adapting and achieving collective function.

neuroscience↗