Uncertainty and Energy Based Loss Guided Semi-Supervised Semantic Segmentation

Rini Smita Thakur, Vinod K Kurmi; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 8024-8034

Semi-supervised (SS) semantic segmentation exploits both labeled and unlabeled images to overcome tedious and costly pixel-level annotation problems. Pseudolabel supervision is one of the core approaches of training networks with both pseudo labels and ground-truth labels. This work uses aleatoric or data uncertainty and energy based modeling in intersection-union pseudo supervised network. The aleatoric uncertainty is modeling the inherent noise variations of the data in a network with two predictive branches. The per-pixel variance parameter obtained from the network gives a quantitative idea about the data uncertainty. Moreover energy-based loss realizes the potential of generative modeling on the downstream SS segmentation task. The aleatoric and energy loss are applied in conjunction with pseudo-intersection labels pseudo-union labels and ground-truth on the respective network branch. The comparative analysis with state-of-the-art methods has shown improvement in performance metrics. The code is availaible at https://visdomlab.github.io/DUEB/.