Detecting Low-Rank Regions in Omnidirectional Images

Zoltan Kato, Gabor Nagy, Martin Humenberger, Gabriela Csurka; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021, pp. 3682-3692

Planar low-rank regions commonly found in man-made environments, can be used to estimate a rectifying homography that provides valuable information about the camera and the 3D plane they observe. Methods to recover such a homography exist, but detection of low-rank regions is largely unsolved, especially for omnidirectional cameras where significant distortions make the problem even more challenging. In this paper we address this problem as follows. First we propose a method to generate a low-rank probability map on an omnidirectional image and use it to build a training set in a self-supervised manner to train deep models to predict low-rank likelihood maps for omnidirectional images. Second, we propose to adapt regular CNN operators to equirectangular images and to combine them seamlessly into a network where each layer preserves the properties of the equirectangular representation. Finally, on the new KITTI360 dataset, we show that the rectifying homography of detected low-rank regions in such predicted maps allows to factorize out the camera-plane pose up to certain ambiguities that can be easily overcome.