Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening

Yule Duan, Xiao Wu, Haoyu Deng, Liang-Jian Deng; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 27738-27747

Currently machine learning-based methods for remote sensing pansharpening have progressed rapidly. However existing pansharpening methods often do not fully exploit differentiating regional information in non-local spaces thereby limiting the effectiveness of the methods and resulting in redundant learning parameters. In this paper we introduce a so-called content-adaptive non-local convolution (CANConv) a novel method tailored for remote sensing image pansharpening. Specifically CANConv employs adaptive convolution ensuring spatial adaptability and incorporates non-local self-similarity through the similarity relationship partition (SRP) and the partition-wise adaptive convolution (PWAC) sub-modules. Furthermore we also propose a corresponding network architecture called CANNet which mainly utilizes the multi-scale self-similarity. Extensive experiments demonstrate the superior performance of CANConv compared with recent promising fusion methods. Besides we substantiate the method's effectiveness through visualization ablation experiments and comparison with existing methods on multiple test sets. The source code is publicly available at https://github.com/duanyll/CANConv.