Flexible Window-based Self-attention Transformer in Thermal Image Super-Resolution

Hongcheng Jiang, Zhiqiang Chen; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3076-3085

The aim of this paper is to improve the resolution of low-quality thermal images obtained from downsampled images afflicted with noise and blur alongside high-resolution visible images to achieve high-resolution thermal imagery. Our proposed method named Flexible Window-based Self-attention Transformer (FW-SAT) operates across global regional and local scales to effectively enhance the fine details in the thermal domain. FW-SAT integrates various attention mechanisms such as channel and spatial attention window-based self-attention and flexible window-based self-attention. Notably flexible window-based self-attention aggregates regional window features based on window-based self-attention while channel and spatial attention mechanisms capture global information. Additionally window-based self-attention is employed to explore local features within the image. We assess the performance of FW-SAT in the PBVS-2024 Thermal Image Super-Resolution Challenge (GTISR) - Track2. Our extensive experiments demonstrate that our proposed approach surpasses state-of-the-art techniques in both qualitative and quantitative evaluations. Code will be available at https://github.com/jianghongcheng/FW-SAT.