-
[pdf]
[supp]
[arXiv]
[bibtex]@InProceedings{Ni_2023_WACV, author = {Ni, Haomiao and Liu, Yihao and Huang, Sharon X. and Xue, Yuan}, title = {Cross-Identity Video Motion Retargeting With Joint Transformation and Synthesis}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2023}, pages = {412-422} }
Cross-Identity Video Motion Retargeting With Joint Transformation and Synthesis
Abstract
In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.
Related Material