A systems vaccinology resource to develop and test computational models of immunity

Shinde, P.; Soldevila, F.; Reyna, J.; Aoki, M.; Rasmussen, M.; Willemsen, L.; Kojima, M.; Ha, B.; Greenbaum, J. A.; Overton, J. A.; Guzman-Orozco, H.; Nili, S.; Orfield, S.; Gygi, J. P.; Antunes, R. d. S.; Sette, A.; Grant, B. J.; Olsen, L. R.; Konstorum, A.; Guan, L.; Ay, F.; Kleinstein, S. H.; Peters, B.

doi:10.1101/2023.08.28.555193

Home/Archive/bioRxiv

bioRxiv · 10.1101/2023.08.28.555193

A systems vaccinology resource to develop and test computational models of immunity

View original source ↗Export metadata ↓

Abstract

Computational models that predict an individuals response to a vaccine offer the potential for mechanistic insights and personalized vaccination strategies. These models are increasingly derived from systems vaccinology studies that generate immune profiles from human cohorts pre- and post-vaccination. Most of these studies involve relatively small cohorts and profile the response to a single vaccine. The ability to assess the performance of the resulting models would be improved by comparing their performance on independent datasets, as has been done with great success in other areas of biology such as protein structure predictions. To transfer this approach to system vaccinology studies, we established a prototype platform that focuses on the evaluation of Computational Models of Immunity to Pertussis Booster vaccinations (CMI-PB). A community resource, CMI-PB generates experimental data for the explicit purpose of model evaluation, which is performed through a series of annual data releases and associated contests. We here report on our experience with the first such dry run for a contest where the goal was to predict individual immune responses based on pre-vaccination multi-omic profiles. Over 30 models adopted from the literature were tested, but only one was predictive, and was based on age alone. The performance of new models built using CMI-PB training data was much better, but varied significantly based on the choice of pre-vaccination features used and the model building strategy. This suggests that previously published models developed for other vaccines do not generalize well to Pertussis Booster vaccination. Overall, these results reinforced the need for comparative analysis across models and datasets that CMI-PB aims to achieve. We are seeking wider community engagement for our first public prediction contest, which will open in early 2024.

Explore related subjects

systems biology Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Shinde, P., Soldevila, F., Reyna, J., Aoki, M., Rasmussen, M., Willemsen, L., Kojima, M., Ha, B., Greenbaum, J. A., Overton, J. A., Guzman-Orozco, H., Nili, S., Orfield, S., Gygi, J. P., Antunes, R. d. S., Sette, A., Grant, B. J., Olsen, L. R., Konstorum, A., Guan, L., Ay, F., Kleinstein, S. H., Peters, B.. 2023-08-29. A systems vaccinology resource to develop and test computational models of immunity. https://doi.org/10.1101/2023.08.28.555193

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Low-fidelity simulator selection for multi-fidelity inference of epithelial vertex models

Vertex models link cell division, neighbour exchange, and intercellular forces to epithelial tissue form through parameters such as junctional line tension and perimeter contractility. Bayesian inference of these parameters from data quantifies their uncertainty, but for stochastic models of proliferating tissues it requires many costly simulations. Multi-fidelity simulation-based inference reduces this cost by training on many runs of a cheaper, less accurate ('low-fidelity') simulator before refining with a few full ('high-fidelity') runs. Low-fidelity versions can be constructed in several ways, for example by stopping simulations early, coarsening the timestep, or averaging fewer replicates, but it is unclear which to choose before committing to costly training. To address this, we introduce PRISM (Paired Ranking Index for Simulator Matching), a screen that compares each candidate with the full simulator on paired runs, scoring agreement in cell-packing statistics against cost. For a vertex model of a proliferating tissue, PRISM flags stopping early as unreliable before training; once trained, this choice can produce confidently wrong parameter estimates. By contrast, a coarser timestep, within the range tested, is cheap and accurate; while fewer replicates are accurate but too costly. Using a screened low-fidelity simulator, multi-fidelity inference outperforms approximate Bayesian computation and neural-network inference trained on full simulations alone, at matched cost and with well-calibrated uncertainty. The same pattern recurs for a second vertex model fitted to published Drosophila pupal-wing imaging data, although there stopping halfway proves safe. Together, our findings suggest that which approximations are safe depends on the model, and that screening them before training makes multi-fidelity inference of tissue mechanics practical.

systems biology↗

bioRxiv2026

Calibrating developmental dynamics from single-cell atlases: lineage paths outrun least-action routes

Single-cell atlases infer developmental dynamics from destructive snapshots, yet how far realized lineage paths depart from such inferred dynamics has never been measured with controls. We introduce LADD (Least-Action Developmental Dynamics), a measurement framework that learns a stochastic developmental vector field from time-series single-cell RNA-seq and computes minimum-action and minimum-time paths, together with a three-layer calibration protocol that separates genuine path deviation from the generic cost of centroid measurement, and we compare its forecasting component with optimal-transport and flow-matching methods. In the lineage-resolved Caenorhabditis elegans atlas, the raw action ratio of observed lineage-centroid trajectories to optimal paths is 3.79, yet forward-simulated ensembles of the learned model itself score 2.07, leaving a genuine excess of about two-fold. In zebrafish (Danio rerio), the observed ratio sits entirely at the model's floor. Three mechanisms that could explain the residual deviation (detours, flow-opposition, and fate-map complexity) are each excluded; what remains is speed, with lineages moving 30-43% faster along the learned flow. We report validity boundaries for temporal resolution, cross-species comparison, and energetic interpretation, delimiting where action-based trajectory comparisons are meaningful.

systems biology↗

bioRxiv2026

Choice of annotation data significantly impacts automated reconstruction of genome-scale metabolic networks

Genome-scale metabolic models are valuable tools for studying metabolism in many applications, such as metabolic engineering or understanding human disease. However, curated models are available for only a limited number of organisms, and generating high-quality models for additional organisms remains a challenge. To help address this challenge, several software tools have been developed that can produce draft models in an automated fashion, but the quality of these draft models lags behind manually curated ones. Thus, there is still a need to improve these tools. Previous work has focused primarily on developing better methods for each step in the reconstruction process, such as gene annotation and gap-filling. Here, we focus on the data that underlies these methods, such as known connections between protein sequence and function. We find that changing the source of reconstruction data has a substantial impact on the contents and predictions of generated models. In fact, the source of data can have a bigger impact than the choice of reconstruction method. Additionally, we show that the reconstruction process is sensitive to the accuracy of gene annotations in the reconstruction data. Finally, we demonstrate the importance of having functional annotation data from an organism that is closely related to the one for which a model is being built. These observations highlight the importance of reconstruction data and identify principles that should be considered when selecting data for reconstruction tools.

systems biology↗