FreqHPT: Frequency-Aware Attention and Flow Fusion for Human Pose Transfer

Liyuan Ma, Tingwei Gao, Haibin Shen, Kejie Huang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 3491-3496

Human pose transfer is a challenging task that synthesizes images in various target poses while preserving the original appearance. This is typically achieved through aligning the source texture and supplementing it to the target pose. However, most of previous alignment methods only rely on either attention or flow, thereby failing to fully leverage distinctive strengths of these two methods. Moreover, the receptive field of these methods is generally limited in supplementation, resulting in the lack of global texture consistency. To address this issue, observing that attention and flow exhibit distinct characteristics in terms of their frequency distribution, Frequency-aware Human Pose Transfer (FreqHPT) is proposed in this paper. FreqHPT investigates the complementarity between attention and flow from the frequency perspective for improving texture-preserving pose transfer. To this end, FreqHPT first transforms the features from attention and flow into the wavelet domain and then fuses them over multi-frequency bands in an adaptive manner. Subsequently, FreqHPT globally refines the fused features in the Fourier space for texture supplement, enhancing the overall semantic consistency. Extensive experiments on the DeepFashion dataset demonstrate the superiority of FreqHPT in generating texture-preserving and realistic pose transfer images.