CACE: Sim-to-Real Indoor 3D Semantic Segmentation via Context-Aware Augmentation and Consistency Enforcement

Tsung-Yu Chen, Luyu Yang, Tzu-Yu Chuang, Shang-Hong Lai; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 8345-8356

Indoor 3D domain adaptation for semantic segmentation is an understudied task. The first unsupervised sim-to-real benchmark was only proposed recently. Existing methods try to modify the source domain data by simulating the occlusion and noise pattern of the target domain. However this methodology unrealistically demands a clear definition of the real-world data patterns and is highly dependent on the simulation quality. In this paper we propose a novel adaptation framework via Context-aware Augmentation and Consistency Enforcement (CACE). Our CACE framework consists of two modules a space and context-aware augmentation module that is invariant of target data pattern and domain gaps and a carefully designed self-supervision module that maximizes the utility of the augmented data. Our CACE surpasses the state-of-the-art method by over 6% on the indoor 3D sim-to-real benchmark 3D-FRONT - ScanNet.