Hebling Vieira, B. | Search bioRxiv

bioRxiv2022

The AI Neuropsychologist: Automatic scoring of memory deficits with deep learning

BackgroundMemory deficits are a hallmark of many different neurological and psychiatric conditions. The Rey-Osterrieth complex figure (ROCF) is the state-of-the-art assessment tool for neuropsychologists across the globe to assess the degree of non-verbal visual memory deterioration. To obtain a score, a trained clinician inspects a patients ROCF drawing and quantifies deviations from the original figure. This manual procedure is time-consuming, slow and scores vary depending on the clinicians experience, motivation and tiredness. MethodsHere, we leverage novel deep learning architectures to automatize the rating of memory deficits. For this, we collected more than 20k hand-drawn ROCF drawings from patients with various neurological and psychiatric disorders as well as healthy participants. Unbiased ground truth ROCF scores were obtained from crowdsourced human intelligence. This dataset was used to train and evaluate a multi-head convolutional neural network. ResultsThe model performs highly unbiased as it yielded predictions very close to the ground truth and the error was similarly distributed around zero. The neural network outperforms both online raters and clinicians. The scoring system can reliably identify and accurately score individual figure elements in previously unseen ROCF drawings, which facilitates explainability of the AI-scoring system. To ensure generalizability and clinical utility, the model performance was successfully replicated in a large independent prospective validation study that was pre-registered prior to data collection. ConclusionsOur AI-powered scoring system provides healthcare institutions worldwide with a digital tool to assess objectively, reliably and time-efficiently the performance in the ROCF test from hand-drawn images.

neuroscience↗

bioRxiv2021

On the prediction of human intelligence from neuroimaging: A systematic review of methods and reporting

Human intelligence is one of the main objects of study in cognitive neuroscience. Reviews and meta-analyses have proved to be fundamental to establish and cement neuroscientific theories on intelligence. The prediction of intelligence using in vivo neuroimaging data and machine learning has become a widely accepted and replicated result. Here, we present a systematic review of this growing area of research, based on studies that employ structural, functional, and/or diffusion MRI to predict human intelligence in cognitively normal subjects using machine-learning. We performed a systematic assessment of methodological and reporting quality, using the PROBAST and TRIPOD assessment forms and 30 studies identified through a systematic search. We observed that fMRI is the most employed modality, resting-state functional connectivity (RSFC) is the most studied predictor, and the Human Connectome Project is the most employed dataset. A meta-analysis revealed a significant difference between the performance obtained in the prediction of general and fluid intelligence from fMRI data, confirming that the quality of measurement moderates this association. The expected performance of studies predicting general intelligence from fMRI was estimated to be r = 0.42 (CI95% = [0.35, 0.50]) while for studies predicting fluid intelligence obtained from a single test, expected performance was estimated as r = 0.15 (CI95% = [0.13, 0.17]). We further enumerate some virtues and pitfalls we identified in the methods for the assessment of intelligence and machine learning. The lack of treatment of confounder variables, including kinship, and small sample sizes were two common occurrences in the literature which increased risk of bias. Reporting quality was fair across studies, although reporting of results and discussion could be vastly improved. We conclude that the current literature on the prediction of intelligence from neuroimaging data is reaching maturity. Performance has been reliably demonstrated, although extending findings to new populations is imperative. Current results could be used by future works to foment new theories on the biological basis of intelligence differences.

neuroscience↗