Fourier Prior-Based Two-Stage Architecture for Image Restoration

Hemkant Nehete, Amit Monga, Partha Kaushik, Brajesh Kumar Kaushik; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 6014-6023

This work presents a novel two stage architecture designed to enhance degraded images affected by environmental factors such as haze blur fog and rain. Despite the dominance of deep Convolutional Neural Networks (CNNs) and Transformers in single image restoration tasks existing methods neglect the intrinsic priors for physical properties of degradation. To enhance the generalization ability of image restoration models we propose Fourier prior based on a key observation that substituting the Fourier amplitude of degraded images with that of clean images effectively mitigates degradation. Therefore amplitude contains degradation information while the phase retains background structures. Consequently a two-stage model is proposed that consists of Amplitude Refinement Unit (ARU) and the Phase Refinement Unit (PRU) that separately restore both amplitude and phase information respectively. ARU and PRU leverage a CNN-Transformer-based architecture to extract local and global features overcoming computational constraints posed by large image sizes in Transformers. Additionally a multi-scale approach in ARU refines amplitude features at coarse and fine levels improving restoration efficiency. Experimental results across multiple image restoration tasks like image deraining dehazing and low-light enhancement indicate that the proposed architecture improved the performance in terms of PSNR SSIM and computational efficiency compared to state-of-the-art Transformer approaches.