FluoNeRF: Fluorescent Novel-View Synthesis under Novel Light Source Colors

Lin Shi, Kengo Matsufuji, Ryo Kawahara, Takahiro Okabe; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 2032-2041

Synthesizing photo-realistic images of a scene from arbitrary viewpoints and under arbitrary lighting environments is one of the important research topics in computer vision and graphics. In this paper we propose a method for synthesizing photo-realistic images of a scene with fluorescent objects from novel viewpoints and under novel lighting colors. In general fluorescent materials absorb light with certain wavelengths and then emit light with longer wavelengths than the absorbed ones in contrast to reflective materials which preserve wavelengths of light. Therefore we cannot reproduce the colors of fluorescent objects under arbitrary lighting colors by combining conventional view synthesis techniques with the white balance adjustment of the RGB channels. Accordingly we extend the novel view synthesis based on the neural radiance fields by incorporating the superposition principle of light; our proposed method captures a sparse set of images of a scene from varying viewpoints and under varying light source colors by using a display-camera system and then synthesize photo-realistic images of the scene without explicitly modeling the geometric and photometric models of the scene. We conduct a number of experiments using real images and confirm the effectiveness of our method.