EDeRF: Updating Local Scenes and Editing Across Fields for Real-Time Dynamic Reconstruction of Road Scene

Zhaoxiang Liang, Wenjun Guo, Yi Yang, Tong Liu; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 3120-3136

Neural Radiance Field(NeRF) offers high reconstruction precision but is time-consuming when training dynamic scenes. Editable NeRF achieves dynamic by editing static scenes, which reduces retraining and has been successfully applied in autonomous driving simulations. However, real-time dynamic road scene reconstruction is challenging due to the high dynamics of real scenes, the lack of depth cameras, and the difficulty in obtaining precise vehicle pose. The primary challenges include the fast and accurate reconstruction of new vehicles entering the scene and their trajectories. We propose EDeRF, a method for real-time dynamic road scene reconstruction from fixed cameras such as traffic surveillance through collaboration of sub-NeRFs and cross-field editing. We decompose the scene space and select key areas to update new vehicles by sharing parameters and local training with sub-fields. These vehicles are then integrated into the complete scene and achieve dynamic motion by warping the sampling rays across different fields, where vehicles' six degrees of freedom(6-DOF) is estimated based on inter-frame displacement and rigid body contact constraints. We have built simulated traffic monitoring scenarios with toll booths in real world and conducted experiments to demonstrate the effectiveness of our method. The results show that EDeRF achieves remarkable results and outperforms comparative methods in terms of real-time capability and accuracy in reconstructing the appearance and movement of newly entered vehicles.