OPDN: Omnidirectional Position-Aware Deformable Network for Omnidirectional Image Super-Resolution

Xiaopeng Sun, Weiqi Li, Zhenyu Zhang, Qiufang Ma, Xuhan Sheng, Ming Cheng, Haoyu Ma, Shijie Zhao, Jian Zhang, Junlin Li, Li Zhang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 1293-1301

Abstract


360deg omnidirectional images have gained research attention due to their immersive and interactive experience, particularly in AR/VR applications. However, they suffer from lower angular resolution due to being captured by fisheye lenses with the same sensor size for capturing planar images. To solve the above issues, we propose a two-stage framework for 360deg omnidirectional image super-resolution. The first stage employs two branches: model A, which incorporates omnidirectional position-aware deformable blocks (OPDB) and Fourier upsampling, and model B, which adds a spatial frequency fusion module (SFF) to model A. Model A aims to enhance the feature extraction ability of 360deg image location characteristics, while Model B further focuses on the high-frequency information of 360deg images. The second stage performs same-resolution enhancement based on the structure of model A with a pixel unshuffle operation. In addition, we collected data from YouTube to improve the fitting ability of the transformer, and created pseudo low-resolution images using a degradation network. Our proposed method achieves superior performance and wins the NTIRE 2023 challenge of 360deg omnidirectional image super-resolution.

Related Material


[pdf] [arXiv]
[bibtex]
@InProceedings{Sun_2023_CVPR, author = {Sun, Xiaopeng and Li, Weiqi and Zhang, Zhenyu and Ma, Qiufang and Sheng, Xuhan and Cheng, Ming and Ma, Haoyu and Zhao, Shijie and Zhang, Jian and Li, Junlin and Zhang, Li}, title = {OPDN: Omnidirectional Position-Aware Deformable Network for Omnidirectional Image Super-Resolution}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2023}, pages = {1293-1301} }