Deep Image Interpolation: A Unified Unsupervised Framework for Pansharpening

Jianhao Gao, Jie Li, Xin Su, Menghui Jiang, Qiangqiang Yuan; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 609-618

Pansharpening, whose aim is to acquire high resolution multispectral data (HRMS) by the fusion of low resolution multispectral data (LRMS) and panchromatic data (PAN), is a specific mission of spatial-spectral fusion in remote sensing field. In recent years, deep learning methods have proved the most feasible methods for pansharpening task. However, these deep learning methods have difficulty in training in an unsupervised manner and become useless when it comes to the condition where no training dataset is available. In this paper, we propose a universal algorithm called deep image interpolation for pansharpening task. The main idea is achieving high-quality fusion results by interpolating low-quality results in a deep neural network. We apply it to two conditions: 1) training a network unsupervisedly when there are enough datasets; 2) optimizing the fusion result in an untrained manner when only a pair of PAN and LRMS are available. Simulation and real-data experiments are conducted on various kinds of satellite data. Quantitative and qualitative evaluation results illustrate that the proposed method outperforms traditional pansharpening methods and even catch up with those supervised methods to some extent.