Personalized Mixture of Experts for Multi-Site Medical Image Segmentation

Md Motiur Rahman, Mohamed Trabelsi, Huseyin Uzunalioglu, Aidan Boyd; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 3172-3184

The sharing of sensitive medical data among institutions presents a significant challenge due to strict privacy regulations the need for robust de-identification processes and the ethical imperative to protect patient confidentiality. Federated Learning (FL) addresses these challenges by enabling institutions to collaboratively train AI models on decentralized data thereby enhancing privacy and security without directly sharing sensitive patient information. However FL requires complex synchronization implementations has costly communication overheads and may fail to capture data heterogeneity across institutions. In this work we propose Personalized Mixture of Local Experts (P-MoLE) a Personalized Federated Learning (PFL) approach that effectively combines predictions from multiple locally trained models in a sample-specific manner. Leveraging both the individuality of each local model and variation across the ensemble P-MoLE learns the profile of each institution's local model and strategically weighs their prediction's contributions to the final segmentation. This approach harnesses the heterogeneity of each institution's unique data to increase the generalization capabilities across all institutions. By each institution sharing only the final models trained locally on the sensitive data no private patient data is exposed and the need for expensive communication infrastructure is removed. Results across two popular multi-institutional medical imaging datasets show P-MoLE achieves state-of-the-art performance without the extensive cooperative effort requirement of previous works. Additionally ablation study results show that P-MoLE is flexible to the number of local models in the ensemble increasing performance over the local models alone in each case.