Physiology-informed regularization enables training of universal differential equation systems for biological applications

Abstract

Systems biology tackles the challenge of understanding the high complexity in the internal regulation of homeostasis in the human body through mathematical modelling. These models can aid in the discovery of disease mechanisms and potential drug targets. However, on one hand the development and validation of knowledge-based mechanistic models is time-consuming and does not scale well with increasing features in medical data. On the other hand, more data-driven approaches such as machine learning models require large volumes of data to produce generalizable models. The integration of neural networks and mechanistic models, forming universal differential equation (UDE) models, enables the automated learning of unknown model terms with less data than the neural network alone. Nevertheless, estimating parameters for these hybrid models remains difficult with sparse data and limited sampling durations that are common in biological applications. In this work, we propose the use of physiology-informed regularization, penalizing biologically implausible model behavior to guide the UDE towards more physiologically plausible regions of the solution space. In a simulation study we show that physiology-informed regularization not only results in a more accurate forecasting of model behaviour, but also supports training with less data. We also applied this technique to learn a representation of the rate of glucose appearance in the glucose minimal model using meal response data measured in healthy people. In that case, the inclusion of regularization reduces variability between UDE-embedded neural networks that were trained from different initial parameter guesses. Author summarySystems biology concerns the modelling and analysis of biological processes, by viewing these as interconnected systems. Modelling is typically done either using mechanistic differential equations that are derived from experiments and known biology, or using machine learning on large biological datasets. While mathematical modelling from biological experiments can provide useful insights with limited data, building and validating these models takes a long time and often requires highly invasive measurements in humans. Efforts to combine this classical technique with machine learning have resulted in a framework termed universal differential equations, where the model equations contain a neural network to describe unknown biological interactions. While these methods have shown success in numerous fields, applications in biology are more challenging due to limited data-availability, high data sparsity. In this work, we have introduced physiology-informed regularization to overcome these instabilities and to constrain the model to biologically plausible behavior. Our results show that by using physiology-informed regularization, we can accurately predict future unseen observations in a simulated example, with much more limited data than a similar model without regularization. Additionally, we show an application of this technique on human data, applying a neural network to learn the appearance of glucose in the blood plasma after a meal.