FCTFANet: A Fused CNN-Transformer Feature Aggregator Network for Image Restoration

Amit Monga, Hemkant Nehete, Partha Kaushik, Tharun Kumar Reddy Bollu, Balasubramanian Raman, Gaurav Sharma; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025, pp. 885-894

Convolutional neural networks (CNNs) have been foundational in deep learning architectures for image processing, and recently, Transformer networks have emerged, bringing further advancements to the field. Unlike CNNs, which extract local spatial features, Transformers are proficient at extracting long-range dependencies and global features within data. However, the Transformer architecture poses significant challenges with large number of parameters and computational demands, particularly for processing large images. To overcome this problem, a Fused CNN-Transformer Feature Aggregator Network (FCTFANet) for image restoration is proposed that leverages CNNs for local feature extraction and Transformers for global feature capture, thereby reducing computational needs. A feature aggregation module fuses CNN and Transformer features from different levels. This aggregation approach facilitates the capture of multi-level features, preserving fine-grained details and high-level contextual information crucial for effective image restoration. Experimental results demonstrate that the proposed approach provides a PSNR improvement of 2.32 dB and 0.86 dB with respect to state-of-the-art CNN and Transformer based image deraining methods with a significant reduction in parameters (17.8x) and computations (3.8x). Additional experimental results with FCTFANet across several image restoration tasks, such as dehazing, denoising, and low-light enhancement, also demonstrate improved performance compared to state-of-the-art approaches.