Empirically-Derived Synthetic Populations to Overcome Small Sample Sizes

Abstract

Limited sample sizes can hinder biomedical research and lead to spurious findings. The objective of this work is to present a new method to generate synthetic populations (SPs) from sparse data samples to aid in modeling developments. Matched case-control data (n=180 pairs) defined the limited samples. Cases and controls were considered as two separate limited samples. Synthetic populations were generated for these observed samples using multivariate unconstrained bandwidth kernel density estimations. We included four continuous variables and one categorical variable for each individual. Bandwidth matrices were determined with Differential Evolution (DE) optimization driven by covariance comparisons. Four synthetic samples (n=180) were constructed from their respective SP for comparison purposes. Similarity between the observed samples with equally sized synthetic-samples was compared under the hypothesis that their sample distributions were the same. Distributions were compared with the maximum mean discrepancy (MMD) test statistic based on a Kernel Two-Sample Test. To evaluate similarity within a modeling context, Principal Component Analysis (PCA) score distributions and residuals were summarized with the distance to the model in X-space (DModX) as additional comparisons.\n\nFour SPs were generated with the optimization procedure. The probability of selecting a replicate when randomly constructing synthetic samples with n=180 was infinitesimally small. The MMD tests indicated that the observed sample distributions were similar to the respective synthetic distributions. For both case and control samples, PCA scores and residuals did not deviate significantly when compared with their respective synthetic samples.\n\nThe reasonableness of this SP generation approach was demonstrated. This approach produced synthetic data at the patient level statistically similar to the observed samples, and thus could be used to generate larger-sized simulated data. The methodology coupled kernel density estimation with DE optimization and deployed novel similarity metrics derived from PCA. The use of large-sized synthetic samples may be a way to overcome sparse datasets. To further develop this approach into a research tool for model building purposes, additional evaluation with increased dimensionality is required; moreover, comparisons with other techniques such as bootstrapping and cross-validation will be required for a complete evaluation.