Biology subjects

Vaickus, L.

Publications and source records attributed to Vaickus, L..

11 recordsLinked to original sources

bioRxiv2024

Integrative Co-Registration of Elemental Imag-ing and Histopathology for Enhanced Spatial Multimodal Analysis of Tissue Sections through TRACE

SummaryElemental imaging provides detailed profiling of metal bioaccumulation, offering more precision than bulk analysis by targeting specific tissue areas. However, accurately identifying comparable tissue regions from elemental maps is challenging, requiring the integration of hematoxylin and eosin (H&E) slides for effective comparison. Facilitating the streamlined co-registration of Whole Slide Images (WSI) and elemental maps, TRACE enhances the analysis of tissue regions and elemental abundance in various pathological conditions. Through an interactive containerized web application, TRACE features real-time annotation editing, advanced statistical tools, and data export, supporting comprehensive spatial analysis. Notably, it allows for comparison of elemental abundances across annotated tissue structures and enables integration with other spatial data types through WSI co-registration. Availability and ImplementationAvailable on the following platforms- GitHub: jlevy44/trace_app, PyPI: trace_app, Docker: joshualevy44/trace_app, Singularity: joshualevy44/trace_app. Contactjoshua.levy@cshs.org Supplementary informationSupplementary data are available.

bioinformatics↗

bioRxiv2023

DeltaAI: Semi-Autonomous Tissue Grossing Measurements and Recommendations using Neural Radiance Fields for Rapid, Complete Intraoperative Histological Assessment of Tumor Margins

Mohs Micrographic Surgery (MMS) aims to excise cutaneous cancer with real-time margin analysis. However, manual tissue grossing and analysis can be inefficient, so we propose DeltaAI, a novel workflow that utilizes Neural Radiance Fields (NeRF) to enable rapid tissue grossing and generate a 3D model in an augmented reality (AR) environment. In our study, we captured 30-second videos of 17 MMS specimens using a photogrammetry turntable and cellphone camera. Preprocessing the tissues with segmentation models, we created a dataset of 923, 360-degree-view, images per video (17 videos). Using COLMAP, we estimated poses for sparse tissue reconstructions and trained the NeRF model for 3D volumetric tissue renderings. The results demonstrated that DeltaAI generated more accurate and complete 360-degree, 3D tissue renderings compared to previous models, while also achieving significantly faster runtimes. Our proposed semi-autonomous NeRF-based workflow has the potential to enhance the speed of MMS specimen processing, measurement, report generation, and margin assessment. It can inform real-time grossing decisions, automate the export of electronic health record data, and facilitate time-efficient and complete cancer excisions. Moreover, DeltaAI can contribute to the wider adoption of AI technology in clinical settings by improving tissue modeling for manual grossing.

pathology↗

bioRxiv2023

Potential to Enhance Large Scale Molecular Assessments of Skin Photoaging through Virtual Inference of Spatial Transcriptomics from Routine Staining

The advent of spatial transcriptomics technologies has heralded a renaissance in research to advance our understanding of the spatial cellular and transcriptional heterogeneity within tissues. Spatial transcriptomics allows investigation of the interplay between cells, molecular pathways and the surrounding tissue architecture and can help elucidate developmental trajectories, disease pathogenesis, and various niches in the tumor microenvironment. Photoaging is the histological and molecular skin damage resulting from chronic/acute sun exposure and is a major risk factor for skin cancer. Spatial transcriptomics technologies hold promise for improving the reliability of evaluating photoaging and developing new therapeutics. Current challenges, including limited focus on dermal elastosis variations and reliance on self-reported measures, can introduce subjectivity and inconsistency. Spatial transcriptomics offer an opportunity to assess photoaging objectively and reproducibly in studies of carcinogenesis and discern the effectiveness of therapies that intervene on photoaging and prevent cancer. Evaluation of distinct histological architectures using highly-multiplexed spatial technologies can identify specific cell lineages that have been understudied due to their location beyond the depth of UV penetration. However, the cost and inter-patient variability using state-of-the-art assays such as the 10x Genomics Spatial Transcriptomics assays limits the scope and scale of large-scale molecular epidemiologic studies. Here, we investigate the inference of spatial transcriptomics information from routine hematoxylin and eosin-stained (H&E) tissue slides. We employed the Visium CytAssist spatial transcriptomics assay to analyze over 18,000 genes at a 50-micron resolution for four patients from a cohort of 261 skin specimens collected adjacent to surgical resection sites for basal and squamous keratinocyte tumors. The spatial transcriptomics data was co-registered with 40x resolution whole slide imaging (WSI) information. We developed machine learning models that achieved a macro-averaged median AUC and F1 score of 0.80 and 0.61 and Spearman coefficient of 0.60 in inferring transcriptomic profiles across the slides, and accurately captured biological pathways across various tissue architectures.

pathology↗

bioRxiv2023

Spatial Omics Driven Crossmodal Pretraining Applied to Graph-based Deep Learning for Cancer Pathology Analysis

Graph-based deep learning has shown great promise in cancer histopathology image analysis by contextualizing complex morphology and structure across whole slide images to make high quality downstream outcome predictions (ex: prognostication). These methods rely on informative representations (i.e., embeddings) of image patches comprising larger slides, which are used as node attributes in slide graphs. Spatial omics data, including spatial transcriptomics, is a novel paradigm offering a wealth of detailed information. Pairing this data with corresponding histological imaging localized at 50-micron resolution, may facilitate the development of algorithms which better appreciate the morphological and molecular underpinnings of carcinogenesis. Here, we explore the utility of leveraging spatial transcriptomics data with a contrastive crossmodal pretraining mechanism to generate deep learning models that can extract molecular and histological information for graph-based learning tasks. Performance on cancer staging, lymph node metastasis prediction, survival prediction, and tissue clustering analyses indicate that the proposed methods bring improvement to graph based deep learning models for histopathological slides compared to leveraging histological information from existing schemes, demonstrating the promise of mining spatial omics data to enhance deep learning for pathology workflows.

pathology↗

bioRxiv2022

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Over 150,000 Americans are diagnosed with colorectal cancer (CRC) every year, and annually over 50,000 individuals will die from CRC, necessitating improvements in screening, prognostication, disease management, and therapeutic options. Tumor metastasis is the primary factor related to the risk of recurrence and mortality. Yet, screening for nodal and distant metastasis is costly, and invasive and incomplete resection may hamper adequate assessment. Signatures of the tumor-immune microenvironment (TIME) at the primary site can provide valuable insights into the aggressiveness of the tumor and the effectiveness of various treatment options. Spatially-resolved transcriptomics technologies offer an unprecedented characterization of TIME through high multiplexing, yet their scope is constrained by cost. Meanwhile, it has long been suspected that histological, cytological and macroarchitectural tissue characteristics correlate well with molecular information (e.g., gene expression). Thus, a method for predicting transcriptomics data through inference of RNA patterns from whole slide images (WSI) is a key step in studying metastasis at scale. In this work, we collected and preprocessed Visium spatial transcriptomics data (17,943 genes at up to 5,000 spots per patient sampled in a honeycomb pattern) from tissue across four stage-III matched colorectal cancer patients. We compare and prototype several convolutional, Transformer, and graph convolutional neural networks to predict spatial RNA patterns under the hypothesis that the transformer and graph-based approaches better capture relevant spatial tissue architecture. We further analyzed the models ability to recapitulate spatial autocorrelation statistics using SPARK and SpatialDE. Overall, results indicate that the transformer and graph-based approaches were unable to outperform the convolutional neural network architecture, though they exhibited optimal performance for relevant disease-associated genes. Initial findings suggest that different neural networks that operate on different scales are relevant for capturing distinct disease pathways (e.g., epithelial to mesenchymal transition). We add further evidence that deep learning models can accurately predict gene expression in whole slide images and comment on understudied factors which may increase its external applicability (e.g., tissue context). Our preliminary work will motivate further investigation of inference for molecular patterns from whole slide images as metastasis predictors and in other applications.

pathology↗

bioRxiv2022

Assessment of Emerging Pretraining Strategies in Interpretable Multimodal Deep Learning for Cancer Prognostication

Deep learning models have demonstrated the remarkable ability to infer cancer patient prognosis from molecular and anatomic pathology information. Studies in recent years have demonstrated that leveraging information from complementary multimodal data can improve prognostication, further illustrating the potential utility of such methods. Model interpretation is crucial for facilitating the clinical adoption of deep learning methods by fostering practitioner understanding and trust in the technology. However, while prior works have presented novel multimodal neural network architectures as means to improve prognostication performance, these approaches: 1) do not comprehensively leverage biological and histomorphological relationships and 2) make use of emerging strategies to "pretrain" models (i.e., train models on a slightly orthogonal dataset/modeling objective) which may aid prognostication by reducing the amount of information required for achieving optimal performance. Here, we develop an interpretable multimodal modeling framework that combines DNA methylation, gene expression, and histopathology (i.e., tissue slides) data, and we compare the performances of crossmodal pretraining, contrastive learning, and transfer learning versus the standard procedure in this context. Our models outperform the existing state-of-the-art method (average 11.54% C-index increase), and baseline clinically driven models. Our results demonstrate that the selection of pretraining strategies is crucial for obtaining highly accurate prognostication models, even more so than devising an innovative model architecture, and further emphasize the all-important role of the tumor microenvironment on disease progression.

pathology↗

bioRxiv2022

Graph neural networks ameliorate potential impacts of imprecise large-scale autonomous immunofluorescence labeling of immune cells on whole slide images

The characteristics of tumor-infiltrating lymphocytes (TIL) are essential in cancer prognostication and treatment through the ability to indicate the tumors capacity to evade the immune system (e.g., as evidenced by nodal involvement). Machine learning technologies have demonstrated remarkable success for localizing TILs, though these methods require extensive curation of manual annotations or restaining procedures that can degrade tissue quality, resulting in imprecise annotation. In this study, we co-registered tissue slides stained for both hematoxylin and eosin (H&E) and immunofluorescence (IF) as means to rapidly perform large-scale annotation of nuclei. We integrated the following approaches to improve the prediction of TILs: 1) minimized tissue degradation on same-section tissue restaining, 2) developed a scoring algorithm to improve the selection of patches for machine learning modeling and 3) utilized a graph neural network deep learning approach to identify relevant contextual features for lymphocyte prediction. Our graph neural network approach accounts for surrounding contextual micro/macro-architecture tissue features to facilitate interpretation of registered IF. The graph neural network compares favorably (F1-score=0.9235, AUROC=0.9462) to two alternative modeling approaches. This study brings insight to the importance of contextual information leveraged from within and around neighboring cells in a nuclei classification workflow, as well as elucidate approaches which enable the rapid generation of large-scale annotations of lymphocytes for machine learning approaches for immune phenotyping. Such approaches can help further interrogate the spatial biology of colorectal cancer tumors and tumor metastasis.

pathology↗

bioRxiv2021

Configuring a federated network of real-world patient health data for multimodal deep learning prediction of health outcomes.

Vast quantities of electronic patient medical data are currently being collated and processed in large federated data repositories. For instance, TriNetX, Inc., a global health research network, has access to more than 300 million patients, sourced from healthcare organizations, biopharmaceutical companies, and contract research organizations. As such, pipelines that are able to algorithmically extract huge quantities of patient data from multiple modalities present opportunities to leverage machine learning and deep learning approaches with the possibility of generating actionable insight. In this work, we present a modular, semi-automated end-to-end machine and deep learning pipeline designed to interface with a federated network of structured patient data. This proof-of-concept pipeline is disease-agnostic, scalable, and requires little domain expertise and manual feature engineering in order to quickly produce results for the case of a user-defined binary outcome event. We demonstrate the pipelines efficacy with three different disease workflows, with high discriminatory power achieved in all cases.

bioinformatics↗

bioRxiv2021

Estimating the Inter- and Intra-Rater Reliability for NASH Fibrosis Staging in the Presence of Bridge Ordinal Ratings with Hierarchical Bridge Category Models

The public health burden of non-alcoholic steatohepatitis (NASH), a liver condition characterized by excessive lipid accumulation and subsequent tissue inflammation and fibrosis, has burgeoned with the spread of western lifestyle habits. Progression of fibrosis into cirrhosis is assessed using histological staging scales (e.g., NASH Clinical Research Network (NASH CRN)). These scales are used to monitor disease progression as well as to evaluate the effectiveness of therapies. However, clinical drug trials for NASH are typically underpowered due to lower than expected inter-/intra-rater reliability, which impacts measurements at screening, baseline, and endpoint. Bridge ratings represent a phenomenon where pathologists assign two adjacent stages simultaneously during assessment and may further complicate these analyses when ad hoc procedures are applied. Statistical techniques, dubbed Bridge Category Models, have been developed to account for bridge ratings, but not for the scenario where multiple pathologists assess biopsies across time points. Here, we develop hierarchical Bayesian extensions for these statistical methods to account for repeat observations and use these methods to assess the impact of bridge ratings on the inter-/intra-rater reliability of the NASH CRN staging scale. We also report on how pathologists may differ in their assignment of bridge ratings to highlight different staging practices. Our findings suggest that Bridge Category Models can capture additional fibrosis staging heterogeneity with greater precision, which translates to potentially higher reliability estimates in contrast to the information lost through ad hoc approaches.

pathology↗

bioRxiv2021

Bridge Category Models: Development of Bayesian Modelling Procedures to Account for Bridge Ordinal Ratings for Disease Staging

Disease grading and staging is accomplished through the assignment of an ordinal rating. Bridge ratings occur when a rater assigns two adjacent categories. Most statistical methodology necessitates the use of a single ordinal category. Consequently, bridge ratings often go unreported in clinical research studies. We propose three methodologies (Expanded, Mixture, and Collapsed) Bridge Category Models, to account for bridge ratings. We perform simulations to examine the impact of our approaches on detecting treatment effects, and comment on a real-world scenario of staging liver biopsies. Results indicate that if bridge ratings are not accounted for, disease staging models may exhibit significant bias and precision loss. All models worked well when they corresponded to the data generating mechanism.

pathology↗

bioRxiv2020

Topological Feature Extraction and Visualization of Whole Slide Images using Graph Neural Networks

Whole-slide images (WSI) are digitized representations of thin sections of stained tissue from various patient sources (biopsy, resection, exfoliation, fluid) and often exceed 100,000 pixels in any given spatial dimension. Deep learning approaches to digital pathology typically extract information from sub-images (patches) and treat the sub-images as independent entities, ignoring contributing information from vital large-scale architectural relationships. Modeling approaches that can capture higher-order dependencies between neighborhoods of tissue patches have demonstrated the potential to improve predictive accuracy while capturing the most essential slide-level information for prognosis, diagnosis and integration with other omics modalities. Here, we review two promising methods for capturing macro and micro architecture of histology images, Graph Neural Networks, which contextualize patch level information from their neighbors through message passing, and Topological Data Analysis, which distills contextual information into its essential components. We introduce a modeling framework, WSI-GTFE that integrates these two approaches in order to identify and quantify key pathogenic information pathways. To demonstrate a simple use case, we utilize these topological methods to develop a tumor invasion score to stage colon cancer.

pathology↗