GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting

Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, Guosheng Lin; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 21476-21485

3D editing plays a crucial role in many areas such as gaming and virtual reality. Traditional 3D editing methods which rely on representations like meshes and point clouds often fall short in realistically depicting complex scenes. On the other hand methods based on implicit 3D representations like Neural Radiance Field (NeRF) render complex scenes effectively but suffer from slow processing speeds and limited control over specific scene areas. In response to these challenges our paper presents GaussianEditor the first 3D editing algorithm based on Gaussian Splatting (GS) a novel 3D representation. GaussianEditor enhances precision and control in editing through our proposed Gaussian semantic tracing which traces the editing target throughout the training process. Additionally we propose Hierarchical Gaussian splatting (HGS) to achieve stabilized and fine results under stochastic generative guidance from 2D diffusion models. We also develop editing strategies for efficient object removal and integration a challenging task for existing methods. Our comprehensive experiments demonstrate GaussianEditor's superior control effective and efficient performance marking a significant advancement in 3D editing.