Contrastive Image Synthesis and Self-Supervised Feature Adaptation for Cross-Modality Biomedical Image Segmentation

Xinrong Hu, Corey Wang, Yiyu Shi; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2337-2346

This work presents a novel framework CISFA (Contrastive Image synthesis and Self-supervised Feature Adaptation) that builds on image domain translation and unsupervised feature adaptation for cross-modality biomedical image segmentation. Different from existing approaches, our method employs a one-sided generative model and incorporates a weighted patch-wise contrastive loss between sampled patches of the input image and the corresponding synthetic image, which serves as shape constraints. Furthermore, we notice that the generated images and input images share similar structural information but are in different modalities. To address this, we enforce contrastive losses on the generated images and the input images to train the encoder of a segmentation model to minimize the discrepancy between paired images in the learned embedding space. Compared with existing works that rely on adversarial learning for feature adaptation, such a method enables the encoder to learn domain-independent features in a more explicit way. We extensively evaluate our methods on segmentation tasks containing CT and MRI images for abdominal cavities and whole hearts. Experimental results show that the proposed framework not only outputs synthetic images with less distortion of organ shapes, but also outperforms state-of-the-art domain adaptation methods.