Attention Guidance Distillation Network for Efficient Image Super-Resolution

Hongyuan Wang, Ziyan Wei, Qingting Tang, Shuli Cheng, Liejun Wang, Yongming Li; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 6287-6296

Over the past decade neural network-based super-resolution techniques have been developed on a large scale with impressive achievements. Many novel solutions have been proposed among which lightweight solutions based on convolutional neural networks have been designed for applications in edge devices. To better realize this application we propose a more lightweight attention guidance distillation network (AGDN). We design the attention guidance distillation block (AGDB) with more efficient space channel and self-attention as the infrastructure of AGDN. Specifically multi-level variance-aware spatial attention (MVSA) is designed to better capture structurally information-rich regions with new multi-scale convolution and local variance alignment. Reallocated contrast-aware channel attention (RCCA) is designed to enhance the processing of common information in all channels while redistributing weights across channels. Sparse global self-attention (SGSA) is introduced for selecting the most useful similarity values for image reconstruction. Extensive experiments demonstrate that AGDN strikes a better balance between performance and complexity compared to other models achieving SOTA performance on several benchmark tests. In addition our AGDN-S ranks first in the FLOPs track and second in the Parameters track of the NTIRE 2024 Efficient SR Challenge. The code is available at https://github.com/daydreamer2024/AGDN.