NL-FFC: Non-Local Fast Fourier Convolution for Image Super Resolution

Abhishek Kumar Sinha, S. Manthira Moorthi, Debajyoti Dhar; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 467-476

Deep neural networks have shown promising results in image super-resolution by learning a complex mapping from low resolution to high resolution image. However, most of the approaches learns to upsample by using convolution in spatial domain and are confined to local features. This results into restricting the receptive field of the network and therefore deteriorates the overall quality of the high-resolution image. To alleviate this issue, we propose a generative model based architecture that learns both local and global features, and fuses them together to generate high quality images. The network uses a non-local attention aided Fast Fourier Convolutions (NL-FFC) to widen the receptive field and learn long-range dependencies. The analyses further show that these Fourier features implicitly provide faster convergence on low frequency components only to learn prior for unobserved high frequency components. The model generalizes well to different datasets. We further investigate the role of non-local attention, and the ratio of local and global features to maximize the performance gain in the ablation study.