Efficient Light Transport Acquisition by Coded Illumination and Robust Photometric Stereo by Dual Photography Using Deep Neural Network

Takafumi Iwaguchi, Hiroshi Kawasaki; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 1165-1173

Light transport is fundamental information to describe both photometric and geometric information of a scene, however, a costly process is required for its acquisition. In this paper, to reduce the sampling time without losing SNR, we propose a technique to use a diffuser as well as a video projector, which projects special patterns designed by deep neural network (DNN). By using the light transport, the scene lit by an arbitrary lighting condition can be synthesized, which will be utilized for various purposes. Among them, photometric stereo (PS) is one important application, which requires multiple images captured under different lighting positions. Although simple PS algorithm cannot be applied to complicated BRDF, we propose a robust PS achieved by using dual photography, which can recover the shape of complicated BRDF, i.e., glitter surfaces of objects. In the experiment, comprehensive evaluations of LT acquisition as well as surface normal estimation using simulation data.