Full-body Human De-lighting with Semi-Supervised Learning

Joshua Weir, Junhong Zhao, Andrew Chalmers, Taehyun Rhee; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 648-664

Removing undesired shading from human images is crucial in supporting various real-world applications. While recent advancements in deep learning-based methods show promise in addressing this challenge, there persists a struggle to accurately separate texture from shading, which often results in unresolved shading artifacts and altered texture patterns. This issue is exacerbated by dataset limitations, such as the lack of diverse real-world clothing styles in realistic datasets and oversimplified assumptions about human reflectance and illumination environments. To address this problem, our paper introduces a novel semi-supervised deep learning method to effectively assemble both real and synthetic data for better disentanglement of texture and shading. We present a global sparsity constraint designed on both labeled and unlabeled data to minimize color variations in the inferred shading map, enhancing texture recovery. By applying this constraint, our method demonstrates improved handling of a broad range of fashion-related textures in the real-world test. Additionally, we address the disparity between real and synthetic data with a novel domain adaptation module to realize effective transfer from synthetic to real images. This module is designed based on the insights of gamma correction, and demonstrates improved shadow removal in real-world images. By integrating these methods, our approach achieves state-of-the-art results, reducing unwanted shading artifacts while maintaining the integrity of underlying textures in real-world scenarios.