Fusion Approaches to Predict Post-Stroke Aphasia Severity from Multimodal Neuroimaging Data

Saurav Chennuri, Sha Lai, Anne Billot, Maria Varkanitsa, Emily J. Braun, Swathi Kiran, Archana Venkataraman, Janusz Konrad, Prakash Ishwar, Margrit Betke; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2644-2653

This paper explores feature selection and fusion methods for predicting the clinical outcome of post-stroke aphasia from medical imaging data. Utilizing a multimodal neuroimaging dataset derived from 55 individuals with chronic aphasia resulting from left-hemisphere lesions following a stroke, two distinct approaches, namely Early Fusion and Late Fusion, were developed using Support Vector Regression or Random Forest regression models for prognosticating patients' functional communication skills measured by Western Aphasia Battery (WAB) test scores. A supervised learning method is proposed to reduce the number of features derived from each imaging modality. The fusion approaches were then applied to find combinations of these reduced feature sets that yield the most accurate WAB predictions. The same nested training/validation/test sets were used for the feature selection and fusion methods. Experiments showed that the best model based on the correlation metric is a Late Fusion RF model (r=0.63), while the best model based on the RMSE is an Early Fusion SVR model (RMSE=16.72). Experiments also revealed several feature set combinations that yielded more accurate predictions than both single-modality feature sets and feature sets that combine all modalities, justifying both fusion and reduction of features derived from multimodal neuroimaging data. It was also found that the percentage of tissue in gray matter regions of the brain, spared by the stroke as identified on structural Magnetic Resonance Imaging, is the single feature set that appeared in all highest ranked feature set combinations of both fusion approaches.