Multi-angle Consistent Generative NeRF with Additive Angular Margin Momentum Contrastive Learning

Hang Zou, Hui Zhang, Yuan Zhang, Hui Ma, Dexin Zhao, Qi Zhang, Qi Li; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 930-939

The NeRF and GAN-based GIRAFFE algorithm has drawn a lot of attention because of its controllable image production capacity. However the consistency of GIRAFFE rendering results from different perspectives of the same object is not stable. The reasons are twofold: First the optimization goal of GIRAFFE is only concerned with whether the generated image resembles the real image or not. Second GIRAFFE could learn knowledge implicitly to complement the feature deformation of large camera angle change which may introduce uncontrollable generation mode resulting in low consistency of the 3D object. This limits its application in fields such as digital person generation and biometric identity. In this paper We introduce an additional Encoder to form a momentum-based Contrastive Learning with the Discriminator of GAN. In addition we propose an AamNCE loss to train our model which introduces an additive angular margin to the positive sample pairs. In brief the proposed framework could be regarded as a new paradigm of GAN and Contrastive Learning. The Contrastive Learning improves the characteristic expression ability of the model and the AamNCE loss makes the category boundaries of the generated images more explicit. The experimental results demonstrate that our method maintains the consistency of face identity well in the multi-angle rotation of the face dataset.