Permutation-calibrated stability discovery under ???? >> ????: A leak-controlled Machine Learning framework identifies candidate proteomics panels in antiseizure medication-related side effects

Abstract

We investigated whether the plasma proteome distinguishes people with epilepsy who report central nervous system (CNS) side effects from antiseizure medications (ASMs) from those who do not. In 161 patients profiled using proximity extension assay-based proteomics Neurology and Inflammation panels ([~]1,447 proteins), we applied an ensemble leak-controlled machine-learning (ML) workflow based on LASSO (linear) and random forest (RF) (non-linear) with repeated nested cross-validation and stability selection. We engineered nested machine-learning workflows that embed permutation-based Monte Carlo p-value estimation directly within model training, enabling statistically calibrated feature discovery under high-dimensional, noisy proteomic data. Discovery phases were explicitly optimized for association and feature robustness, not prediction. The RF yielded a 61-protein candidate panel, for which an "exploratory nested RF" model achieved strong internal discrimination of CNS side effects (AUROC [~] 0.92, 95% CI [~] 0.86-0.96). The LASSO yielded a three-protein candidate panel all of which overlapped with those of the RF (SMOC2, TANK and IMPG1). Because per-protein testing across all 1447 proteins produced false discovery rates (FDRs) close to 1, we performed post-hoc, data-driven routed per-protein inference restricted to the 61-protein panel, identifying 13 proteins with FDR <0.1. Network and pathway analyses on the 61-protein panel highlighted immune, autoimmune and vascular-inflammation pathways (e.g. cytokine networks, JAK-STAT, T-cell-mediated responses), suggesting that pre-existing immune and inflammatory may modulate vulnerability to ASM-related CNS side effects. Technically, our contribution presents a resampling-based stability statistics and FDR control despite p >> n and weak global discrimination. This framework is model-agnostic and directly portable to other low-sample, high-dimensional, noisy omics settings by replacing the base learner (e.g., LASSO/RF/boosting) while keeping the same leakage-safe resampling and permutation-calibrated stability machinery to prioritize robust biomarkers over optimistic predictive accuracy. By explicitly separating robust discovery from post-selection exploratory modeling, the workflow provides a reproducible template for generating candidate panels when standard whole-proteome multiple testing is underpowered. Author SummaryWe studied whether patterns in blood proteins can distinguish people with epilepsy who experience central nervous system side effects from anti-seizure medications from those who do not. Working with 161 patients and about 1,447 measured proteins, we faced a common challenge in modern biology: far more measurements than patients, substantial noise, and strong correlations among proteins. To address this, we built a reproducible analysis template that prioritizes reliable discovery over optimistic prediction. Our approach combines two complementary machine-learning models and repeatedly tests them on held-out patients to prevent information from "leaking" from the test data into training. We then use repeated re-sampling and label shuffling to estimate how often each protein is selected just by chance, which lets us compute calibrated p-values and false discovery rates for machine-learning feature selection. This makes the resulting candidate protein panel easier to interpret statistically and less sensitive to random fluctuations in small datasets. Using this framework, we identified a 61-protein candidate panel and a smaller overlapping set of three proteins highlighted by both models, and then performed targeted follow-up testing within the panel. Because the workflow is model-agnostic and leak-controlled, it can be reused in many other omics studies with limited sample sizes.