Multimodal Dementia Screening from Brain Magnetic Resonance Imaging and Conversational Speech

Early detection of Alzheimer's dementia is crucial for effective symptomatic treatment. Herein, we explore the opportunities of dementia screening based on automatically derived biomarkers from Magnetic Resonance Imaging (MRI) and spoken communication. In particular, we compare the screening performance of unimodal MRI-based or speech-based markers with their multimodal counterparts. We use structural MRI data and interview recordings from the Interdisciplinary Longitudinal Study on Adult Development and Aging to analyze state and predictive screening between participants with and without cognitive impairment. We assess the effect of varying training set sizes on unimodal screening performance and compare screening performance of unimodal data with multimodal data, which are combined through Early, Late and Tensor Fusion. For both state and predictive screening, classifiers that combined both modalities with the Early Fusion method outperformed the single modalities, achieving an Unweighted Average Recall of 83.2% and 74.9%, respectively. On the evaluated dataset, markers derived from speech are just as useful for predictive screening as markers derived from MRI.