Reconstructing Spectral Images From RGB-Images Using a Convolutional Neural Network

Tarek Stiebel, Simon Koppers, Philipp Seltsam, Dorit Merhof; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 948-953

Recovering high-dimensional spectral images taken with spectrally low-dimensional camera systems, in the extreme case RGB-images, has been of great interest for a variety of applications. An accurate spectral reconstruction is typically required to either achieve a better color accuracy or to improve object recognition/classification tasks. Almost all published work to date aims at performing a mapping from individual camera signals towards the corresponding spectrum. However, it might be beneficial to consider not only single pixels, but also contextual information. Here, we propose a convolutional neural network architecture that learns a mapping from RGB- to spectral images. We trained the network on the largest hyper-spectral data set available to date and analyzed the influence of different error metrics as loss functions. An objective evaluation of the performance in comparison to state of the art spectral reconstruction techniques is given by participating in the NTIRE 2018 challenge on spectral reconstruction.