Reviving Poor Object Segmentations in OOD Medical Images using Variational-Deep-PCA Modeling on Segmentation Maps with Sampling-Free Learning

Jimut B. Pal, Shantanu Welling, Himali Saini, Suyash P. Awate; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 9346-9355

For object segmentation in medical images deep neural networks (DNNs) typically perform poorly on out-of-distribution (OOD) images stemming from the large variability in image-acquisition equipment and protocols across sites. However compared to such variability in the acquired medical images we observe that the variability in the underlying object-segmentation maps is far lower. Thus we propose a novel DNN framework to model this variability in segmentation maps and leverage it to revive poor segmentations produced by existing DNNs on OOD images. Our DNN framework (i) learns the principal modes of variation in a class of segmentation maps (ii) models each segmentation map using a low-dimensional mixture-of-modes latent representation on a simplex (iii) enables sampling-free variational learning and uncertainty estimation and (iv) trains using small in-distribution image sets. When OOD-image segmentations are extremely poor we propose a novel human-in-the-loop method needing minuscule human intervention. Results using 6 publicly-available datasets and 8 existing DNN segmenters show the benefits of our framework in OOD-image object segmentation.