My3DGen: A Scalable Personalized 3D Generative Model

Luchao Qi, Jiaye Wu, Annie N. Wang, Shengze Wang, Roni Sengupta; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 961-972

In recent years generative 3D face models (e.g. EG3D) have been developed to tackle the problem of synthesizing photo-realistic faces. However these models are often unable to capture facial features unique to each individual highlighting the importance of personalization. Some prior works have shown promise in personalizing generative face models but these studies primarily focus on 2D settings. Also these methods require both fine-tuning and storing a large number of parameters for each user posing a hindrance to achieving scalable personalization. Another challenge of personalization is the limited number of training images available for each individual which often leads to overfitting when using full fine-tuning methods. Our proposed approach My3DGen generates a personalized 3D prior of an individual using as few as 50 training images. My3DGen allows for novel view synthesis semantic editing of a given face (e.g. adding a smile) and synthesizing novel appearances all while preserving the original person's identity. We decouple the 3D facial features into global features and personalized features by freezing the pre-trained EG3D and training additional personalized weights through low-rank decomposition. As a result My3DGen introduces only 240K personalized parameters per individual leading to a 127x reduction in trainable parameters compared to the 30.6M required for fine-tuning the entire parameter space. Despite this significant reduction in storage our model preserves identity features without compromising the quality of downstream applications both quantitatively and qualitatively.