NeuroMechFly 2.0, a framework for simulating embodied sensorimotor control in adult Drosophila
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Until now, such models, including NeuroMechFly for the adult fly, Drosophila melanogaster, have primarily been used to investigate motor control. Far less studied with realistic body models is how the brain and motor systems work together to perform hierarchical sensorimotor control. Here we present NeuroMechFly v2, a framework that expands Drosophila neuromechanical modeling by enabling visual and olfactory sensing, ascending motor feedback, and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by first constructing biologically inspired locomotor controllers that use ascending motor feedback to perform path integration and head stabilization. Then, we add visual and olfactory sensing to this controller and train it using reinforcement learning to perform a multimodal navigation task in closed loop. Finally, we illustrate more biorealistic modeling in two ways: our model navigates a complex odor plume using a Drosophila odor taxis strategy, and it uses a connectome-constrained visual system network to follow another simulated fly. With this framework, NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.