Biology subjects

Piccolo, S. R.

Publications and source records attributed to Piccolo, S. R..

18 recordsLinked to original sources

bioRxiv2026

Impacts of batch effects on the performance of machine learning classifiers across multiple studies

Precision medicine relies on accurate and generalizable predictions for patients across the spectrum of human diversity. Because capturing biological heterogeneity requires large sample sizes, researchers must often aggregate data from several experimental batches or independent studies. This integration allows for greater statistical power and diversity than a single study could provide, while avoiding the costs of generating massive new -omics datasets. Predictive models trained on these aggregated data are theoretically better equipped to detect subtle patterns that generalize to new data. However, this potential is frequently undermined by "batch effects"--systematic technical artifacts that can bias model training to predict experimental batches and shadow meaningful biological conditions. Models trained on data with batch effects can exhibit substantially degraded performance when applied to data from new batches. Statistical adjustment methods can mitigate these artifacts while preserving biological signals. To ensure these adjustments actually facilitate generalization, we emphasize the use of external, independent cohorts for rigorous validation. This chapter examines how batch effects impact predictions and compares various adjustment methods.

bioinformatics↗

bioRxiv2025

Using semantic search to find publicly available gene-expression datasets

Millions of high-throughput, molecular datasets have been shared in public repositories. have been shared in public repositories. Researchers can reuse such data to validate their own findings and explore novel questions. A frequent goal is to find multiple datasets that address similar research topics and to either combine them directly or integrate inferences from them. However, a major challenge is finding relevant datasets due to the vast number of candidates, inconsistencies in their descriptions, and a lack of semantic annotations. This challenge is first among the FAIR principles for scientific data. Here we focus on dataset discovery within Gene Expression Omnibus (GEO), a repository containing 100,000s of data series. GEO supports queries based on keywords, ontology terms, and other annotations. However, reviewing these results is time-consuming and tedious, and it often misses relevant datasets. We hypothesized that language models could address this problem by summarizing dataset descriptions as numeric representations (embeddings). Assuming a researcher has previously found some relevant datasets, we evaluated the potential to find additional relevant datasets. For six human medical conditions, we used 30 models to generate embeddings for datasets that human curators had previously associated with the conditions and identified other datasets with the most similar descriptions. This approach was often, but not always, more effective than GEOs search engine. Our top-performing models were trained on general corpora, used contrastive-learning strategies, and used relatively large embeddings. Our findings suggest that language models have the potential to improve dataset discovery, perhaps in combination with existing search tools.

genomics↗

bioRxiv2024

A Comprehensive Meta-Analysis of Breast Cancer Gene Expression

BackgroundTriple-negative breast cancers (TNBC) occur more frequently in African Americans and are associated with worse outcomes when compared to other subtypes of breast cancer. These cancers lack expression of estrogen receptor (ER), progesterone receptor (PR) and human epidermal growth factor receptor 2 (HER2) and have limited treatment options. To shed light on mechanisms behind these differences and suggest novel treatments, we used a meta-analytic approach to identify gene expression differences in breast tumors for people with self-reported African or European ancestry; additionally, we compared gene expression levels based on ER, PR, HER2 and TNBC status. MethodsAfter gathering and standardizing gene expression data and metadata from 106 datasets (representing 27,000 samples), we identified genes that were expressed differently between these groups via random-effects meta-analyses. To evaluate the robustness of these gene lists, we devised a novel computational methodology that uses cross validation and classification. We also computed overlaps between the most significant genes and known signaling pathways. ResultsUsing a false discovery rate threshold of 0.05, we identified genes that are known to play a significant role in their respective breast cancer subtypes (e.g., ESR1 for ER status and ERBB2 for HER2 status), thus confirming the validity of our findings. We also discovered genes that have not been reported previously and may be new targets for breast cancer therapy. GATA3, CA12, TBC1D9, XBP1 and FOXA1 were among the most significant genes for ER, PR, and TNBC. However, none of these genes overlapped with HER2 status, supporting prior research that HER2 tumors are mechanistically different from endocrine breast cancers. The genes identified from the race meta-analysis--including DNAJC15, HLA-DPA1, STAP2, CEP68, MOGS--have not been associated previously with race-specific breast-cancer outcomes, highlighting a potential area of further research. ConclusionsWe have carried out a large meta-analysis of breast cancer gene expression data, identifying novel genes that may serve as potential biomarkers for breast cancer in diverse populations. We have also developed a computational method that identifies gene sets small enough to be analyzed and explored in future studies. This method has the potential to be applied to other cancers.

cancer biology↗

bioRxiv2023

Identifying images in the biology literature that are problematic for people with a color-vision deficiency

To help maximize the impact of scientific journal articles, authors must ensure that article figures are accessible to people with color-vision deficiencies (CVDs), which affect up to 8% of males and 0.5% of females. We evaluated images published in biology-and medicine-oriented research articles between 2012 and 2022. Most included at least one color contrast that could be problematic for people with deuteranopia ("deuteranopes"), the most common form of CVD. However, spatial distances and within-image labels frequently mitigated potential problems. Initially, we reviewed 4,964 images from eLife, comparing each against a simulated version that approximated how it might appear to deuteranopes. We identified 636 (12.8%) images that we determined would be difficult for deuteranopes to interpret. Our findings suggest that the frequency of this problem has decreased over time and that articles from cell-oriented disciplines were most often problematic. We used machine learning to automate the identification of problematic images. For hold-out test sets from eLife (n = 879) and PubMed Central (n = 1,191), a convolutional neural network classified the images with areas under the precision-recall curve of 0.75 and 0.38, respectively. We created a Web application (https://bioapps.byu.edu/colorblind_image_tester); users can upload images, view simulated versions, and obtain predictions. Our findings shed new light on the frequency and nature of scientific images that may be problematic for deuteranopes and motivate additional efforts to increase accessibility.

bioinformatics↗

bioRxiv2023

Annotated Compendium of 102 Breast Cancer Gene-Expression Datasets

Transcriptomic data from breast-cancer patients are widely available in public repositories. However, before a researcher can perform statistical inferences or make biological interpretations from such data, they must find relevant datasets, download the data, and perform quality checks. In many cases, it is also useful to normalize and standardize the data for consistency and to use updated genome annotations. Additionally, researchers need to parse and interpret metadata: clinical and demographic characteristics of patients. Each of these steps requires computational and/or biomedical expertise, thus imposing a barrier to reuse for many researchers. We have identified and curated 102 publicly available, breast-cancer datasets representing 17,151 patients. We created a reproducible, computational pipeline to download the data, perform quality checks, renormalize the raw gene-expression measurements (when available), assign gene identifiers from multiple databases, and annotate the metadata against the National Cancer Institute Thesaurus, thus making it easier to infer semantic meaning and compare insights across datasets. We have made the curated data and pipeline freely available for other researchers to use. Having these resources in one place promises to accelerate breast-cancer research, enabling researchers to address diverse types of questions, using data from a variety of patient populations and study contexts.

genomics↗

bioRxiv2023

TidyGEO: Preparing analysis-ready datasets from Gene Expression Omnibus

TidyGEO is a Web-based tool for downloading, tidying, and reformatting data series from Gene Expression Omnibus (GEO). As a freely accessible repository with data from over 4 million biological samples across more than 4,000 organisms, GEO provides diverse opportunities for secondary research. Transcriptomic data are most common in GEO, but other measurement types are also prevalent, including DNA methylation levels, genotypes, and chromatin-accessibility profiles. GEOs diversity and expansiveness present opportunities and challenges. Although scientists may find assay data relevant to a given research question, most analyses require sample annotations, such as a samples treatment group, disease subtype, or age. In GEO, such annotations are stored alongside assay data in delimited, text-based files. However, the structure and semantics of the annotations vary widely from one series to another, and many annotations are not useful for analysis purposes. Thus, every GEO series must be tidied before it can be analyzed. Manual approaches may be used, but these are error prone and take time away from other research tasks. Custom computer scripts can be written, but many scientists lack the computational expertise to create such scripts. To address these challenges, we created TidyGEO, which supports essential data-cleaning tasks for sample-level annotations, such as selecting informative columns, renaming columns, splitting or merging columns, standardizing data values, and filtering samples. Additionally, users can integrate annotations with assay data, restructure assay data, and generate code that enables others to reproduce these steps. The source code for TidyGEO is at https://github.com/srp33/TidyGEO.

bioinformatics↗

bioRxiv2021

The DNA methylation landscape of five pediatric-tumor types

Fewer DNA mutations have been identified in pediatric tumors than adult tumors, suggesting that alternative tumorigenic mechanisms, including aberrant DNA methylation, may play a prominent role in pediatric tumors. Methylation is an epigenetic process of regulating gene expression in which methyl groups are attached to DNA molecules, often in promoter regions. In Wilms tumors and acute myeloid leukemias, increased levels of epigenetic silencing have been associated with worse patient outcomes. However, to date, researchers have studied methylation primarily in adult tumors and for specific genes but not on a pan-pediatric cancer scale. We addressed these gaps first by aggregating methylation data from 309 noncancerous samples and establishing baseline expectations for each gene. Even though these samples represent diverse tissue types and population ancestral groups, methylation levels were highly consistent for most genes. Second, we compared tumor methylation levels against these baseline values for five pediatric cancer types--Wilms tumors, clear cell sarcomas of the kidney, rhabdoid tumors, neuroblastomas, and osteosarcomas. Hypermethylation was more common than hypomethylation--as many as 11.8% of genes were hypermethylated in a given tumor, compared to a maximum of 4.8% for hypomethylated genes. For each cancer type, genes with the highest variance exhibited consistently divergent methylation patterns for distinct patient subsets. We evaluated whether genomic and epigenomic abnormalities contribute to pediatric tumorigenesis in a mutually exclusive manner but did not find evidence of this phenomenon. Furthermore, even though oncogenes are commonly upregulated in tumors, and tumor-suppressor genes are commonly downregulated in tumors, we did not find statistical evidence that methylation drives such patterns on a broad scale in pediatric tumors.

genomics↗

bioRxiv2021

Benchmarking 50 classification algorithms on 50 gene-expression datasets

By classifying patients into subgroups, clinicians can provide more effective care than using a uniform approach for all patients. Such subgroups might include patients with a particular disease subtype, patients with a good (or poor) prognosis, or patients most (or least) likely to respond to a particular therapy. Diverse types of biomarkers have been proposed for assigning patients to subgroups. For example, DNA variants in tumors show promise as biomarkers; however, tumors exhibit considerable genomic heterogeneity. As an alternative, transcriptomic measurements reflect the downstream effects of genomic and epigenomic variations. However, high-throughput technologies generate thousands of measurements per patient, and complex dependencies exist among genes, so it may be infeasible to classify patients using traditional statistical models. Machine-learning classification algorithms can help with this problem. However, hundreds of classification algorithms exist--and most support diverse hyperparameters--so it is difficult for researchers to know which are optimal for gene-expression biomarkers. We performed a benchmark comparison, applying 50 classification algorithms to 50 gene-expression datasets (143 class variables). We evaluated algorithms that represent diverse machine-learning methodologies and have been implemented in general-purpose, open-source, machine-learning libraries. When available, we combined clinical predictors with gene-expression data. Additionally, we evaluated the effects of performing hyperparameter optimization and feature selection in nested cross-validation folds. Kernel- and ensemble-based algorithms consistently outperformed other types of classification algorithms; however, even the top-performing algorithms performed poorly in some cases. Hyperparameter optimization and feature selection typically improved predictive performance, and univariate feature-selection algorithms outperformed more sophisticated methods. Together, our findings illustrate that algorithm performance varies considerably when other factors are held constant and thus that algorithm selection is a critical step in biomarker studies. Author Summary

bioinformatics↗

bioRxiv2021

Simplifying the development of portable, scalable, and reproducible workflows

Command-line software plays a critical role in biology research. However, processes for installing and executing software differ widely. The Common Workflow Language (CWL) is a community standard that addresses this problem. Using CWL, tool developers can formally describe a tools inputs, outputs, and other execution details in a manner that fosters use of shared computational methods and reproducibility of complex analyses. CWL documents can include instructions for executing tools inside software containers--isolated, operating-system environments. Accordingly, CWL tools are portable--they can be executed on diverse computers--including personal workstations, high-performance clusters, or the cloud. This portability enables easier adoption of bioinformatics pipelines. CWL supports workflows, which describe dependencies among tools and using outputs from one tool as inputs to others. To date, CWL has been used primarily for batch processing of large datasets, especially in genomics. But it can also be used for analytical steps of a study. This article explains key concepts about CWL and software containers and provides examples for using CWL in biology research. CWL documents are text-based, so they can be created manually, without computer programming. However, ensuring that these documents confirm to the CWL specification may prevent some users from adopting it. To address this gap, we created ToolJig, a Web application that enables researchers to create CWL documents interactively. ToolJig validates information provided by the user to ensure it is complete and valid. After creating a CWL tool or workflow, the user can create "input-object" files, which store values for a particular invocation of a tool or workflow. In addition, ToolJig provides examples of how to execute the tool or workflow via a workflow engine.

bioinformatics↗

bioRxiv2020

Predicting drug sensitivity of cancer cells based on DNA methylation levels

Cancer cell lines, which are cell cultures developed from tumor samples, represent one of the least expensive and most studied preclinical models for drug development. Accurately predicting drug response for a given cell line based on molecular features may help to optimize drug-development pipelines and explain mechanisms behind treatment responses. In this study, we focus on DNA methylation profiles as one type of molecular feature that is known to drive tumorigenesis and modulate treatment responses. Using genome-wide, DNA methylation profiles from 987 cell lines from the Genomics of Drug Sensitivity in Cancer database, we applied machine-learning algorithms to evaluate the potential to predict cytotoxic responses for eight anti-cancer drugs. We compared the performance of five classification algorithms and four regression algorithms that use diverse methodologies, including tree-, probability-, kernel-, ensemble-, and distance-based approaches. For both types of algorithm, we artificially subsampled the data to varying degrees, aiming to understand whether training models based on relatively extreme outcomes would yield improved performance. We also performed an information-gain analysis to examine which genes were most predictive of drug responses. Finally, we used tumor data from The Cancer Genome Atlas to evaluate the feasibility of predicting clinical responses in humans based on models derived from cell lines. When using classification or regression algorithms to predict discrete or continuous responses, respectively, we consistently observed excellent predictive performance when the training and test sets both consisted of cell-line data. However, classification models derived from cell-line data failed to generalize effectively for tumors.

bioinformatics↗

bioRxiv2020

Topsentinol L Trisulfate, a new Marine Natural Product, that Targets Basal-like and Claudin-low Breast Cancers

BackgroundBreast cancer is a heterogeneous disease. Genomic studies have revealed five different intrinsic subtypes - Luminal A, Luminal B, HER2-enriched, Claudin-low, and Basal-like. Patients diagnosed with Basal-like or Claudin-low breast cancer (BL-CL) suffer from poor prognosis and limited treatment options. Hence, there is an urgent need to identify a new therapeutic lead that can benefit patients with BL-CL breast cancer. MethodsWe used a step-wise screening approach to screen 2778 HP20 fractions from our Marine Invertebrate Compound Library (MICL) to identify compounds that specifically target BL-CL breast cancer. We also performed biochemical investigations to study the effect of the compound in the signaling pathway. Finally, we generated a drug response gene-expression signature and projected it against a human tumor panel of 12 different cancer types to identify other cancer types sensitive to the compound. ResultsWe identified a previously unreported trisulfated sterol, topsentinol L trisulfate (TLT) that exhibits increased efficacy against BL-CL relative to Luminal/HER2+ breast cancer. Biochemical investigation of the effects of TLT on BL-CL revealed its ability to inhibit activation of AMPK and CHK1 and promote activation of p38. The importance of targeting AMPK and CHK1 in BL-CL was validated by treating a panel of breast cancer cell lines with known small molecule inhibitors of AMPK (Dorsomorphin) and CHK1 (Ly2603618) and recording the increased effectiveness against BL-CL compared to Luminal/HER2+ breast cancer. The TLT sensitivity gene-expression signature identified breast and bladder cancer as the most sensitive to TLT while glioblastoma multiforme as least sensitive. ConclusionsOur study identified TLT, a previously uncharacterized trisulfated sterol, as a potential therapeutic selective against BL-CL. Our results also showed that inhibition of AMPK and/or CHK1 might be an effective therapy in BL-CL.

cancer biology↗

bioRxiv2019

ShinyLearner: A containerized benchmarking tool for machine-learning classification of tabular data

Classification algorithms assign observations to groups based on patterns in data. The machine-learning community have developed myriad classification algorithms, which are employed in diverse life-science research domains. When applying such algorithms, researchers face the challenge of deciding which algorithm(s) to apply in a given research domain. Algorithm choice can affect classification accuracy dramatically, so it is crucial that researchers optimize these choices based on empirical evidence rather than hearsay or anecdotal experience. In benchmark studies, multiple algorithms are applied to multiple datasets, and the researcher examines overall trends. In addition, the researcher may evaluate multiple hyperparameter combinations for each algorithm and use feature selection to reduce data dimensionality. Although software implementations of classification algorithms are widely available, robust benchmark comparisons are difficult to perform when researchers wish to compare algorithms that span multiple software packages.\n\nProgramming interfaces, data formats, and evaluation procedures differ across software packages; and dependency conflicts may arise during installation. To address these challenges, we created ShinyLearner, an open-source project for integrating machine-learning packages into software containers. ShinyLearner provides a uniform interface for performing classification, irrespective of the library that implements each algorithm, thus facilitating benchmark comparisons. In addition, ShinyLearner enables researchers to optimize hyperparameters and select features via nested cross validation; it tracks all nested operations and generates output files that make these steps transparent. ShinyLearner includes a Web interface to help users more easily construct the commands necessary to perform benchmark comparisons. ShinyLearner is freely available at https://github.com/srp33/ShinyLearner.

bioinformatics↗

bioRxiv2019

Markers of BRCAness in breast cancer

BackgroundMutations in BRCA1 and BRCA2 cause deficiencies in homologous recombination repair (HR), resulting in repair of DNA double-strand breaks by the alternative non-homologous end-joining pathway, which is more error prone. HR deficiency of breast tumors is important because it is associated with better responses to platinum salt therapies and PARP inhibitors. Among other consequences of HR deficiency are characteristic somatic-mutation signatures and gene-expression patterns. The term "BRCAness" describes tumors that harbor an HR defect but have no detectable germline mutation in BRCA1 or BRCA2. A better understanding of the genes and molecular events associated with BRCAness could provide mechanistic insights and guide development of targeted treatments. MethodsUsing data from The Cancer Genome Atlas (TCGA) for 1101 breast-cancer patients, we identified individuals with a germline mutation, somatic mutation, homozygous deletion, and/or hypermethylation event in BRCA1, BRCA2, and 59 other cancer-predisposition genes. Based on the assumption that BRCAness events would have similar downstream effects on tumor biology as BRCA1/BRCA2 germline mutations, we quantified these effects based on somatic-mutation signatures and gene-expression profiles. We reduced the dimensionality of the somatic-mutation signatures and expression data and used a statistical resampling approach to quantify similarities between patients who had a BRCA1/BRCA2 germline mutation, another type of aberration in BRCA1 or BRCA2, or any type of aberration in one of the other genes. ResultsSomatic-mutation signatures of tumors having a somatic mutation, homozygous deletion, or hypermethylation event in BRCA1/BRCA2 (n = 80) were generally similar to each other and to tumors from BRCA1/BRCA2 germline carriers (n = 44). Additionally, somatic-mutation signatures of tumors with germline or somatic events in ATR (n = 16) and BARD1 (n = 8) showed high similarity to tumors from BRCA1/BRCA2 carriers. Other genes also showed high similarity but only for a small number of events or for a single event type. Tumors with germline mutations or hypermethylation of BRCA1 had relatively similar gene-expression profiles and overlapped considerably with the Basal-like subtype; but the transcriptional effects of the other events lacked consistency. ConclusionsOur findings confirm previously known relationships between molecular signatures and germline or somatic events in BRCA1/BRCA2 and suggest additional genes that may be considered for inclusion in the definition of BRCAness.

cancer biology↗

bioRxiv2019

Coordinate-based mapping of tabular data enables fast and scalable queries

MotivationBiologists commonly store data in tabular form with observations as rows, attributes as columns, and measurements as values. Due to advances in high-throughput technologies, the sizes of tabular datasets are increasing. Some datasets contain millions of rows or columns. To work effectively with such data, researchers must be able to efficiently extract subsets of the data (using filters to select specific rows and retrieving specific columns). However, existing methodologies for querying tabular data do not scale adequately to large datasets or require specialized tools for processing. We sought a methodology that would overcome these challenges and that could be applied to an existing, text-based format. ResultsIn a systematic benchmark, we tested 10 techniques for querying simulated, tabular datasets. These techniques included a delimiter-splitting method, the Python pandas module, regular expressions, object serialization, the awk utility, and string-based indexing. We found that storing the data in fixed-width formats provided excellent performance for extracting data subsets. Because columns have the same width on every row, we could pre-calculate column and row coordinates and quickly extract relevant data from the files. Memory mapping led to additional performance gains. A limitation of fixed-width files is the increased storage requirement of buffer characters. Compression algorithms help to mitigate this limitation at a cost of reduced query speeds. Lastly, we used this methodology to transpose tabular files that were hundreds of gigabytes in size, without creating temporary files. We propose coordinate-based, fixed-width storage as a fast, scalable methodology for querying tabular biological data. Contactstephen_piccolo@byu.edu

bioinformatics↗

bioRxiv2017

Combined analysis of genome sequencing and RNA-motifs reveals novel damaging non-coding mutations in human tumors

A major challenge in cancer research is to determine the biological and clinical significance of somatic mutations in non-coding regions. This has been studied in terms of recurrence, functional impact, and association to individual regulatory sites, but the combinatorial contribution of mutations to common RNA regulatory motifs has not been explored. We developed a new method, MIRA, to perform the first comprehensive study of significantly mutated regions (SMRs) affecting binding sites for RNA-binding proteins (RBPs) in cancer. Extracting signals related to RNA-related selection processes and using RNA sequencing data from the same samples we identified alterations in RNA expression and splicing linked to mutations on RBP binding sites. We found SRSF10 and MBNL1 motifs in introns, HNRPLL motifs at 5 UTRs, as well as 5 and 3 splice-site motifs, among others, with specific mutational patterns that disrupt the motif and impact RNA processing. MIRA facilitates the integrative analysis of multiple genome sites that operate collectively through common RBPs and can aid in the interpretation of non-coding variants in cancer. MIRA is available at https://github.com/comprna/mira.

cancer biology↗

bioRxiv2017

Curated compendium of human transcriptional biomarker data

Genome-wide transcriptional profiles provide broad insights into cellular activity. One important use of such data isto identify relationships between transcription levels and patient outcomes. These translational insights can guide the development of biomarkers for predicting outcomes in clinical settings. Over the past decades, data from many translational-biomarker studies have been deposited in public repositories, enabling other scientists to reuse the data in follow-up studies. However, data-reuse efforts require considerable time and expertise because transcriptional data are generated using heterogeneous profiling technologies, preprocessed using diverse normalization procedures, and annotated in non-standard ways. To address this problem, we curated a compendium of 45 translational-biomarker datasets from the public domain. To increase the datas utility, we reprocessed the raw expression data using a standard computational pipeline and standardized the clinical annotations in a fully reproducible manner (see osf.io/ssk3t). We believe these data will be particularly useful to researchers seeking to validate gene-level findings or to perform benchmarking studies--for example, to compare and optimize machine-learning algorithms ability to predict biomedical outcomes.

cancer biology↗

bioRxiv2017

Classifying cancer genome aberrations by their mutually exclusive effects on transcription

BackgroundMalignant tumors are typically caused by a conglomeration of genomic aberrations--including point mutations, small insertions, small deletions, and large copy-number variations. In some cases, specific chemotherapies and targeted drug treatments are effective against tumors that harbor certain genomic aberrations. However, predictive aberrations (biomarkers) have not been identified for many tumors. One way to address this problem is to examine the downstream, transcriptional effects of genomic aberrations and to identify characteristic patterns. Even though two tumors harbor different genomic aberrations, the transcriptional effects of those aberrations may be similar. These patterns could be used to inform treatment choices.\n\nResultsWe hypothesized that by grouping genomic aberrations according to their downstream transcriptional effects, we could identify drug-repurposing candidates that coincide with prior knowledge. To evaluate this hypothesis, we developed a computational pipeline that uses supervised machine learning to identify similarities among mutually exclusive genomic aberrations based on their transcriptional effects. We used data from 9300 tumors across 25 cancer types from The Cancer Genome Atlas to evaluate our approach. Our pipeline recapitulates known relationships in cancer pathways, and it identifies gene pairs known to predict responses to the same treatments.\n\nConclusionWe show that transcriptional data offer promise as a way to group genomic aberrations according to their downstream effects, and these groupings recapitulate known relationships. Our approach shows potential to help pharmacologists and clinical trialists narrow the search space for candidate gene/drug associations, including for rare mutations, and for identifying potential drug-repurposing opportunities.

genomics↗

bioRxiv2016

A community-based collaboration to build prediction models for short-term discontinuation of docetaxel in metastatic castration-resistant prostate cancer patients

BackgroundDocetaxel has a demonstrated survival benefit for metastatic castration-resistant prostate cancer (mCRPC). However, 10-20% of patients discontinue docetaxel prematurely because of toxicity-induced adverse events, and managing risk factors for toxicity remains an ongoing challenge for health care providers and patients. Prospective identification of high-risk patients for early discontinuation has the potential to assist clinical decision-making and can improve the design of more efficient clinical trials. In partnership with Project Data Sphere (PDS), a non-profit initiative facilitating clinical trial data-sharing, we designed an open-data, crowdsourced DREAM (Dialogue for Reverse Engineering Assessments and Methods) Challenge for developing models to predict early discontinuation of docetaxel\n\nMethodsData from the comparator arms of four phase III clinical trials in first-line mCRPC were obtained from PDS, including 476 patients treated with docetaxel and prednisone from the ASCENT2 trial, 598 patients treated with docetaxel, prednisone/prednisolone, and placebo in the VENICE trial, 526 patients treated with docetaxel, prednisone, and placebo in the MAINSAIL trial, and 528 patients treated with docetaxel and placebo in the ENTHUSE 33 trial. Early discontinuation was defined as treatment stoppage within three months due to adverse treatment effects. Over 150 clinical features including laboratory values, medical history, lesion measures, prior treatment, and demographic variables were curated and made freely available for model building for all four trials. The ASCENT2, VENICE, and MAINSAIL trial data sets formed the training set that also included patient discontinuation status. The ENTHUSE 33 trial, with patient discontinuation status hidden, was used as an independent validation set to evaluate model performance. Prediction performance was assessed using area under the precision-recall curve (AUPRC) and the Bayes factor was used to compare the performance between prediction models.\n\nResultsThe frequency of early discontinuation was similar between training (ASCENT2, VENICE, and MAINSAIL) and validation (ENTHUSE 33) sets, 12.3% versus 10.4% of docetaxel-treated patients, respectively. In total, 34 independent teams submitted predictions from 61 different models. AUPRC ranged from 0.088 to 0.178 across submissions with a random model performance of 0.104. Seven models with comparable AUPRC scores (Bayes factor [≤]; 3) were observed to outperform all other models. A post-challenge analysis of risk predictions generated by these seven models revealed three distinct patient subgroups: patients consistently predicted to be at high-risk or low-risk for early discontinuation and those with discordant risk predictions. Early discontinuation events were two-times higher in the high-versus low-risk subgroup and baseline clinical features such as presence/absence of metastatic liver lesions, and prior treatment with analgesics and ACE inhibitors exhibited statistically significant differences between the high- and low-risk subgroups (adjusted P < 0.05). An ensemble-based model constructed from a post-Challenge community collaboration resulted in the best overall prediction performance (AUPRC = 0.230) and represented a marked improvement over any individual Challenge submission. A\n\nFindingsOur results demonstrate that routinely collected clinical features can be used to prospectively inform clinicians of mCRPC patients risk to discontinue docetaxel treatment early due to adverse events and to the best of our knowledge is the first to establish performance benchmarks in this area. This work also underscores the \"wisdom of crowds\" approach by demonstrating that improved prediction of patient outcomes is obtainable by combining methods across an extended community. These findings were made possible because data from separate trials were made publicly available and centrally compiled through PDS.

bioinformatics↗