No Shadow Left Behind: Removing Objects and Their Shadows Using Approximate Lighting and Geometry

Edward Zhang, Ricardo Martin-Brualla, Janne Kontkanen, Brian L. Curless; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 16397-16406

Abstract


Removing objects from images is a challenging technical problem that is important for many applications, including mixed reality. For believable results, the shadows that the object casts should also be removed. Current inpainting-based methods only remove the object itself, leaving shadows behind, or at best require specifying shadow regions to inpaint. We introduce a deep learning pipeline for removing a shadow along with its caster. We leverage rough scene models in order to remove a wide variety of shadows (hard or soft, dark or subtle, large or thin) from surfaces with a wide variety of textures. We train our pipeline on synthetically rendered data, and show qualitative and quantitative results on both synthetic and real scenes.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Zhang_2021_CVPR, author = {Zhang, Edward and Martin-Brualla, Ricardo and Kontkanen, Janne and Curless, Brian L.}, title = {No Shadow Left Behind: Removing Objects and Their Shadows Using Approximate Lighting and Geometry}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2021}, pages = {16397-16406} }