Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation

Dong Zhao, Shuang Wang, Qi Zang, Licheng Jiao, Nicu Sebe, Zhun Zhong; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 23416-23427

We study source-free unsupervised domain adaptation (SFUDA) for semantic segmentation which aims to adapt a source-trained model to the target domain without accessing the source data. Many works have been proposed to address this challenging problem among which uncertainty based self-training is a predominant approach. However without comprehensive denoising mechanisms they still largely fall into biased estimates when dealing with different domains and confirmation bias. In this paper we observe that pseudo-label noise is mainly contained in unstable samples in which the predictions of most pixels undergo significant variations during self-training. Inspired by this we propose a novel mechanism to denoise unstable samples with stable ones. Specifically we introduce the Stable Neighbor Denoising (SND) approach which effectively discovers highly correlated stable and unstable samples by nearest neighbor retrieval and guides the reliable optimization of unstable samples by bi-level learning. Moreover we compensate for the stable set by object-level object paste which can further eliminate the bias caused by less learned classes. Our SND enjoys two advantages. First SND does not require a specific segmentor structure endowing its universality. Second SND simultaneously addresses the issues of class domain and confirmation biases during adaptation ensuring its effectiveness. Extensive experiments show that SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings. In addition SND can be easily integrated with other approaches obtaining further improvements. The source code will be publicly available.