Multispectral Reconstruction From Reference Objects in the Scene

Nirit Nussbaum Hoffer, Tomer Michaeli; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 0-0

Hyperspectral imaging methods typically require dedicated cameras with extra optical elements (prisms, fibers, lenslet arrays), thus making them expensive and cumbersome to deploy. In this paper we explore a drastically different hyperspectral imaging approach, which requires no special optical components and can thus be used with any conventional camera. The idea is to place a reference object with a known spectrum (e.g. a black mask) within the field of view and to exploit the chromatic dependence of the Point Spread Function (PSF), in order to solve for the spectra of all other parts of the scene. We prove mathematically that chromatic-dependent blur cues alone are insufficient for fully recovering the spectrum of each pixel, even if the locations of edges in the (sharp) image are precisely known. Yet, we show that knowing the spectra at some of the pixels fully resolves this inherent ambiguity. We present an algorithm for solving the spectrum-from-reference inverse problem and illustrate its effectiveness through simulations as well as in a simple real world experiment