Enhancing Multimedia Applications by Removing Dynamic Objects in Neural Radiance Fields

XianBen Yang, Tao Wang, He Liu, Yi Jin, Congyan Lang, Yidong Li; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 2070-2086

Neural Radiance Fields (NeRF) are at the forefront of view synthesis technology, renowned for their versatility and ease of implementation across various applications. However, their integration into multimedia environments faces challenges: objects occlude background information during motion, which usually compromises the quality of reconstructions. In this paper, we present a novel framework to exclude dynamic interference from NeRF scenes, enhancing the practicability in multimedia applications. Our method leverages perceptual optimization, informed by image quality assessment (IQA), and employs text-guided 2D image inpainting to address view synthesis inaccuracies. Furthermore, we propose a new and challenging dataset of real-world scenes to address the lack of evaluation ground truth for dynamic object inpainting in scenes. Experimental results show that our method significantly outperforms existing methods in terms of appearance metrics for the task of removing dynamic objects from scenes.