Evaluation and comparison of methods for neuronal parameter optimization using the Neuroptimus software framework
Finding optimal parameters for detailed neuronal models is a ubiquitous challenge in neuroscientific research. Recently, manual model tuning has been replaced by automated parameter search using a variety of different tools and methods. However, using most of these software tools and choosing the most appropriate algorithm for a given optimization task require substantial technical expertise, which prevents the majority of researchers from using these methods effectively. To address these issues, we developed a generic platform (called Neuroptimus) that allows users to set up neural parameter optimization tasks via a graphical interface, and to solve these tasks using a wide selection of state-of-the-art parameter search methods implemented by five different Python packages. Neuroptimus also offers several features to support more advanced usage, including the ability to run most algorithms in parallel, which allows it to take advantage of high-performance computing architectures. We used the common interface provided by Neuroptimus to conduct a detailed comparison of more than twenty different algorithms (and implementations) on six distinct benchmarks that represent typical scenarios in neuronal parameter search. We quantified the performance of the algorithms in terms of the best solutions found and in terms of convergence speed. We identified several algorithms, including covariance matrix adaptation evolution strategy and particle swarm optimization, that consistently found good solutions in all of our use cases. By contrast, some other algorithms including all local search methods provided good solutions only for the simplest use cases, and failed completely on more complex problems. Finally, we created an online database that allows uploading, querying and analyzing the results of optimization runs performed by Neuroptimus, which enables all researchers to update and extend the current benchmarking study. The tools and analysis we provide should aid members of the neuroscience community to apply parameter search methods more effectively in their research. Author summaryModel fitting is a widely used method in scientific research. It involves tuning the free parameters of a model until its output best matches the corresponding experimental data. Finding the optimal parameter combination can be a difficult task for more complex models with many unknown parameters, and a large variety of different approaches have been proposed to solve this problem. However, setting up a parameter search task and employing an efficient algorithm for its solution requires considerable technical expertise. We have developed a software framework that helps users solve this task, focusing on the domain of detailed models of single neurons. Our open-source software, called Neuroptimus, has a graphical interface that guides users through the steps of setting up a parameter optimization task, and allows them to select from more than twenty different algorithms to solve the problem. We have also compared the performance of these algorithms on a set of six parameter search tasks that are typical in neuroscience, and identified several algorithms that delivered consistently good performance. Finally, we designed and implemented a website that allows users to view and analyze our results and to add their own results to the database.