Schmitt, M. S. | Search bioRxiv

bioRxiv2026

Learning functional groups in complex microbiomes

From soil to the gut, communities composed of thousands of microbes perform functions such as carbon sequestration and immune system regulation. Here, we introduce a data-driven approach that explains how community function can be traced to just a few groups of microbes or genes. In gut communities, our neural-network based clustering algorithm correctly recovers known functional groups. In the ocean metagenome, it distills ~500 gene modules down to three sparse groups highlighting survival strategies at different depths. In soils, it distills ~ 4400 bacterial species into two groups that enter a mathematical model of nitrate metabolism. By combining interpretable ML with strain isolation and sequencing experiments, we connect the metabolic specialization of each group to community-wide responses to perturbations. This integrated approach yields simple structure-function maps of microbiomes, allowing the discovery of molecular mechanisms underlying human and environmental health. More broadly, we illustrate how to do function-informed dimensionality reduction in biology. O_FIG O_LINKSMALLFIG WIDTH=158 HEIGHT=200 SRC="FIGDIR/small/709366v1_ufig1.gif" ALT="Figure 1"> View larger version (41K): org.highwire.dtl.DTLVardef@1f76c74org.highwire.dtl.DTLVardef@f13bdcorg.highwire.dtl.DTLVardef@19107c6org.highwire.dtl.DTLVardef@9558f7_HPS_FORMAT_FIGEXP M_FIG O_FLOATNOGraphical AbstractC_FLOATNO An integrated ML and experimental pipeline to discover functional groups and their dynamics in complex microbiomes and beyond. (a) First, our Soft Clustering Function Informed (SCiFI) algorithm identifies functional groups directly from species abundances data using neural networks. Crucially, the learned functional groups are informed by a chosen community function. (b) Descriptions of the system in terms of the original high-dimensional abundances of individual species are prohibitively complex (left). By contrast, a description in terms of the few functional groups identified by SCiFI leads to a simple structure-function map (right). (c) Next, the identified groups can be directly input as variables into predictive mathematical models for the dynamics of the community. (d) The last step of our pipeline relies on the identified groups comprising only a small number of species. This sparsity enables targeted experiments that interrogate isolated species (e.g. with whole-genome sequencing or phenotyping) shedding light on the mechanistic underpinnings of the structure-function map with potential applications beyond microbiomes, from gene expression to neuronal activity. C_FIG

microbiology↗

bioRxiv2024

Information theory for data-driven model reduction in physics and biology

Model reduction is the construction of simple yet predictive descriptions of the dynamics of many-body systems in terms of a few relevant variables. A prerequisite to model reduction is the identification of these relevant variables, a task for which no general method exists. Here, we develop a systematic approach based on the information bottleneck to identify the relevant variables, defined as those most predictive of the future. We elucidate analytically the relation between these relevant variables and the eigenfunctions of the transfer operator describing the dynamics. Further, we show that in the limit of high compression, the relevant variables are directly determined by the slowest-decaying eigenfunctions. Our information-based approach indicates when to optimally stop increasing the complexity of the reduced model. Furthermore, it provides a firm foundation to construct interpretable deep learning tools that perform model reduction. We illustrate how these tools work in practice by considering uncurated videos of atmospheric flows from which our algorithms automatically extract the dominant slow collective variables, as well as experimental videos of cyanobacteria colonies in which we discover an emergent synchronization order parameter. Significance StatementThe first step to understand natural phenomena is to intuit which variables best describe them. An ambitious goal of artificial intelligence is to automate this process. Here, we develop a framework to identify these relevant variables directly from complex datasets. Very much like MP3 compression is about retaining information that matters most to the human ear, our approach is about keeping information that matters most to predict the future. We formalize this insight mathematically and systematically answer the question of when to stop increasing the complexity of minimal models. We illustrate how interpretable deep learning tools built on these ideas reveal emergent collective variables in settings ranging from satellite recordings of atmospheric fluid flows to experimental videos of cyanobacteria colonies.

biophysics↗