Color-cued Efficient Densification Method for 3D Gaussian Splatting

Sieun Kim, Kyungjin Lee, Youngki Lee; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 775-783

Many variants of Neural Radiance Fields (NeRF) have been explored in pursuit of high-quality results with reasonable data size and real-time rendering speed. 3D Gaussian Splatting (3DGS) gained popularity due to its ability to render quality images in real-time; however it still faces challenges with large data sizes. Meanwhile the densification process of 3DGS plays a large role in deciding the quality and the data size of a model. Hence it is crucial to devise a densification method that can populate Gaussians efficiently so that quality can be enhanced and fewer Gaussians are used. An efficient densification method that results in fewer Gaussians can also promote efficiency in training time GPU memory usage and rendering speed. Hence we propose a novel efficient densification method based on color cues aiming to achieve a more compact Gaussian model without sacrificing image quality. By expanding the original 3DGS densification scheme we identify weaknesses in the original method that lead to redundant Gaussians and compromise quality. In contrast to the original approach which relies solely on the 2D position gradient our method additionally leverages the spherical harmonics (SH) gradient to consider color cues. This approach resolves the inefficiencies of the original densification by aligning with the expanded scheme. Our method achieves at least 9x data size reduction with increased perceptual quality accompanied by additional efficiencies in training time GPU memory usage and rendering speed.