Deep RGB-driven Learning Network for Unsupervised Hyperspectral Image Super-resolution

Zhe Liu, Xian-Hua Han; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2022, pp. 220-234

Hyperspectral (HS) images are used in many fields to improve the analysis and understanding performance of captured scenes, as they contain a wide range of spectral information. However, the spatial resolution of hyperspectral images is usually very low, which limits their wide applicability in real tasks. To address the problem of low spatial resolution, super-resolution (SR) methods for hyperspectral images (HSI) have attracted widespread interest, which aims to mathematically generate high spatial resolution hyperspectral (HR-HS) images by combining degraded observational data: low spatial resolution hyperspectral (LR-HS) images and high resolution multispectral or RGB (HR-MS/RGB) images. Recently, paradigms based on deep learning have been widely explored as an alternative to automatically learn the inherent priors for the latent HR-HS images. These learning-based approaches are usually implemented in a fully supervised manner and require large external datasets including degraded observational data: LR-HS/HR-RGB images and corresponding HR-HS data, which are difficult to collect, especially for HSI SR scenarios. Therefore, in this study, a new unsupervised HSI SR method is proposed that uses only the observed LR-HS and HR-RGB images without any other external samples. Specifically, we use an RGB-driven deep generative network to learn the desired HR-HS images using a encoding-decoding-based network architecture. Since the observed HR-RGB images have a more detailed spatial structure and may be more suitable for two-dimensional convolution operations, we employ the observed HR-RGB images as input to the network as a conditional guide and adopt the observed LR-HS/HR-RGB images to formulate the loss function that guides the network learning. Experimental results on two HS image datasets show that our proposed unsupervised approach provides superior results compared to the SoTA deep learning paradigms.