Image-Based Relighting With 5-D Incident Light Fields

Shinnosuke Oya, Takahiro Okabe; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 3031-3038

In this paper, we propose a method for image-based relighting with 5-D incident light fields: 4 DoF of the position and direction and 1 DoF of the color of an incident ray. Specifically, we illuminate a scene with various rays by using a two-layer 5 DoF lighting system consisting of a rear-projection display and a transmissive LC panel, and synthesize images under desired 5-D incident light fields by combining the images captured under those rays. Our proposed method efficiently acquires the required images by using coded illumination; it reduces the number of captured images and the measurement time, and enhances their SNRs. In addition, we propose a method for removing the effects of the black offsets due to the projector and the LC panel in the two-layer setup. The experimental results using the prototype system show that our method enables us to synthesize photo-realistic images of scenes where wavelength-dependent phenomena such as fluorescence are observed.