Semi-Stereo: A Universal Stereo Matching Framework for Imperfect Data via Semi-supervised Learning

Xin Yue, Zongqing Lu, Xiangru Lin, Wenjia Ren, Zhijing Shao, Haonan Hu, Yu Zhang, Qingmin Liao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 646-655

Data matters in deep-learning-based binocular stereo matching. Obtaining a perfect dataset for stereo matching is hard and thus imperfect data is common in existing benchmark datasets such as KITTI ETH3D and Middlebury. The imperfectness typically has two forms: sparse-labeled data or even unlabeled data. Current stereo matching networks ignore the supervision from these imperfect data itself even the semi-supervised networks often suffer from confirmation bias in the predictions. Besides current methods lack a unified solution to utilize the supervision signal from those imperfect data. To mitigate this research gap we propose Semi-Stereo the first unified stereo matching framework empowered by the teacher-student paradigm where the teacher and the student networks are trained in a mutual-beneficial manner. To explore the rich knowledge in imperfect data we propose a consistency regularization module with weak-strong augmentation strategies. Further in order for the teacher to provide more reliable pseudo labels we design a confidence module powered by left-right consistency (LRC) check and disparity distribution entropy (DDE). Extensive experiments demonstrate Semi-Stereo produces accurate and consistent predictions in untrained semantic regions and improves the performance of baseline networks in multiple tasks including domain adaptation and domain generalization.