Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On

Igor Santesteban, Nils Thuerey, Miguel A. Otaduy, Dan Casas; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 11763-11773

Abstract


We propose a new generative model for 3D garment deformations that enables us to learn, for first time, a data-driven method for virtual try-on that effectively addresses garment-body collisions. In contrast to existing methods that require an undesirable postprocessing step to fix garment-body interpenetrations at test time, our approach directly outputs 3D garment configurations that do not collide with the underlying body. Key to our success is a new canonical space for garments that removes pose-and-shape deformations already captured by a new diffused human body model, which extrapolates body surface properties such as skinning weights and blendshapes to any 3D point. We leverage this representation to train a generative model with a novel self-supervised collision term that learns to reliably solve garment-body interpenetrations. We extensively evaluate and compare our results with recently proposed data-driven methods, and show that our method is the first to successfully address garment-body contact in unseen body shapes and motions, without compromising the realism and detail.

Related Material


[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Santesteban_2021_CVPR, author = {Santesteban, Igor and Thuerey, Nils and Otaduy, Miguel A. and Casas, Dan}, title = {Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2021}, pages = {11763-11773} }