Leveraging Large Language Models to Extract Prognostic Pathology Features in Ewing Sarcoma

Abstract

BackgroundCurrent risk stratification for Ewing sarcoma relies heavily on clinical factors such as metastatic status, failing to capture histologic heterogeneity as a potential prognostic indicator. Although pathology reports contain rich biological data, this information remains locked in unstructured narrative text, limiting large-scale retrospective analyses. We aimed to validate the utility of Large Language Models (LLMs) for scalable data abstraction and to identify prognostic histologic features from a large multi-institutional cohort. MethodsWe conducted a retrospective cohort study using data from six Childrens Oncology Group (COG) clinical trials. We utilized an LLM-based pipeline (OpenAI o3) to extract structured variables--including immunohistochemical (IHC) markers and CD99 staining patterns--from digitized, Optical Character Recognition (OCR)-processed pathology reports. Extraction accuracy was validated against a human-annotated ground truth (n=200) and cross-validated against senior experts (n=48). We assessed the association between extracted features and Overall Survival (OS) using Kaplan-Meier analysis and multivariable Cox proportional hazards regression, adjusting for metastatic status. FindingsWe analyzed 931 diagnostic pathology reports from 185 institutions spanning over 21-years. The LLM achieved a weighted average accuracy of 94% across 17 IHC markers; in a cross-validation subset, the LLM outperformed human annotators (weighted average accuracy over 15 IHC markers: LLM o3: 98.1%, a pediatric resident 91.4%, and a pediatric oncologist 95.9%). Survival analysis identified Neuron-Specific Enolase (NSE) and S100 as significant prognostic biomarkers. After adjusting for metastatic status, NSE positivity was associated with significantly inferior survival (HR 2.15, 95% CI 1.15-4.02, p=0.016); this risk was most pronounced in patients with non-metastatic disease (HR 5.64, p=0.0055). Conversely, S100 positivity was associated with improved survival (HR 0.58, 95% CI 0.34-1.00, p=0.046). InterpretationLLM-assisted extraction of pathology variables is highly accurate and scalable, capable of unlocking "dark data" from historical clinical trials. We identified NSE as a potent risk factor and S100 as a protective marker in Ewing sarcoma, particularly in localized disease. These findings suggest that AI-derived histologic data can refine risk stratification and, if validated, warrant inclusion in future prospective trials. Research in ContextO_ST_ABSEvidence before this studyC_ST_ABSWe searched PubMed and Google Scholar for articles published up to December 2025, using terms such as "Ewing sarcoma", "prognosis", "risk stratification", "large language models", "natural language processing", and "pathology report extraction". Current risk stratification for Ewing sarcoma relies predominantly on clinical variables, specifically the presence of metastatic disease and tumor size. While histologic heterogeneity is well-documented, it is rarely incorporated into risk models because extracting structured data from narrative pathology reports is labor-intensive. Small, single-institution studies have suggested potential prognostic roles for markers like Neuron-Specific Enolase (NSE) or S100, but results have been inconsistent and limited by small sample sizes. Furthermore, while Large Language Models (LLMs) have demonstrated potential for extracting clinical data, their application to "rescue" data from legacy, multi-institutional clinical trial documents (scanned PDF images) for rare disease biomarker discovery has not been validated at scale. Added value of this studyTo our knowledge, this is the largest study to utilize AI to extract histologic data from Ewing sarcoma pathology reports, aggregating 931 patients from six distinct Childrens Oncology Group (COG) clinical trials spanning over 21 years. We validated an LLM-based pipeline that converted noisy, optically character-recognized text from diverse institutions into structured data with 98.1% accuracy, outperforming human annotators. This scalable approach revealed that NSE positivity is a significant independent risk factor for mortality (HR=2.15), particularly in patients with non-metastatic disease where it confers a more than 5-fold increased risk of death. Conversely, we identified S100 positivity as a protective factor associated with improved survival (HR=0.58). Implications of all the available evidenceThis study demonstrates that LLMs can reliably unlock "dark data" from historical clinical trials, rendering vast archives of unstructured medical documents accessible for retrospective analysis. The identification of NSE and S100 as robust prognostic biomarkers suggests that these widely available immunohistochemical stains provide valuable information beyond standard diagnostic information. These findings support the integration of automated data extraction tools in research workflows and suggest that NSE and S100 status should be considered in the design of future risk-stratified clinical trials for Ewing sarcoma.