Multi-scale Attention-Based Inclination Angles Estimation for Panoramic Camera

Yuhao Shan, Heyu Chen, Jiaying Zhang, Shigang Li, Jianfeng Li; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1322-1330

Images taken by panoramic cameras in the upright posture can give viewers a better sense and make the downstream panoramic image-based computer vision tasks easier. To estimate the inclination angles of panoramic camera we proposed a simple but elegant panoramic image-based network which combines the advantages of geometry-based and deep-learning-based methods. First a backbone network with five down-sampling layers is designed to focus on the local distortion features. Then since non-upright panoramic images have highly uniform geometric distortion for the same camera inclination angles a multi-scale attention module is proposed for the first time which can weigh each pixel on the feature maps of the backbone network and allows the network to focus on the global and shallow geometric features. Moreover apart from angle loss pixel-level image loss is introduced in our network for the inclination angles estimation task to allow the network to compensate for pixel deviations during training. The experiments show that our method overcomes other leading state-of-the-art methods in this field.