Spatial-Angular Multi-Scale Mechanism for Light Field Spatial Super-Resolution

Chen Gao, Youfang Lin, Song Chang, Shuo Zhang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 1961-1970

Light Field (LF) cameras are promising due to their ability to capture both spatial and angular information of scenes. However, the trade-off between spatial and angular resolution significantly limits the real-world applications. In this paper, we propose a spatial-angular multi-scale decoupling network to reconstruct high-resolution LF images. Considering the epipolar geometry, we propose a spatial-angular multi-scale processing approach to explore the correspondence of sub-pixel information with different disparity ranges between sub-aperture images in LFs. We extract sub-pixel information from various dimensions and fuse it to generate global high-frequency details. Finally, we combine upsampled low-frequency and high-frequency details to generate high resolution results. To further filter the correct interpolation information, we use the shear operation to change the disparity range of the LF images and fine-tune the results. Experimental results on synthetic and real-world datasets demonstrate that our method outperforms other state-of-the-art methods in visual and numerical evaluations, especially on datasets with small disparity ranges. Furthermore, our approach fully considers the epipolar geometry of the LF image, enabling us to recover information that better maintains the imaging consistency of the LF.