GraVITON: Graph based garment warping with attention guided inversion for Virtual-tryon

Sanhita Pathak, Vinay Kaushik, Brejesh Lall; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2024, pp. 573-588

Virtual try-on, a rapidly evolving field in computer vision, is transforming e-commerce by improving customer experiences through precise garment warping and seamless integration onto the human body. Existing methods such as TPS and flow address the garment warping, but overlook the finer contextual details. In this paper, we introduce a novel graph based warping technique which emphasizes the value of context in garment flow. Our graph based warping module generates warped garment as well as a coarse person image, which is utilised by a simple refinement network to give a coarse virtual tryon image. We then exploit a latent diffusion model to generate the final tryon, treating garment transfer as an inpainting task. The diffusion model incorporates a Decoupled Garment Attention Adaptor(DGAA) for attention based diffusion inversion of visual and textual information. Our method, validated on VITON-HD and Dresscode datasets, showcases substantial state-of-the-art qualitative and quantitative results showing considerable improvement in garment warping, texture preservation, and overall realism.