Shadow Removal Refinement via Material-Consistent Shadow Edges

Shilin Hu, Hieu Le, ShahRukh Athar, Sagnik Das, Dimitris Samaras; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 2631-2641

Shadow boundaries can be confused with material boundaries as both exhibit sharp changes in luminance or contrast within a scene. However shadows do not modify the intrinsic color or texture of surfaces. Therefore on both sides of shadow edges traversing regions with the same material the original color and texture should be the same if the shadow is removed properly. These shadow/shadow-free pairs are very useful but difficult-to-collect supervision signals. The crucial contribution of this paper is to learn how to identify those shadow edges that traverse material-consistent regions and how to use them as self-supervision for shadow removal refinement during test time. To achieve this we fine-tune SAM an image segmentation foundation model to produce a shadow-invariant segmentation and then extract material-consistent shadow edges by comparing the SAM segmentation with the shadow mask. Utilizing these shadow edges we introduce color- and texture-consistency losses to enhance the shadow removal process. We demonstrate the effectiveness of our method in improving shadow removal results on more challenging in-the-wild images outperforming the state-of-the-art shadow removal methods. Additionally we propose a new metric and an annotated dataset for evaluating the performance of shadow removal methods without the need for paired shadow/shadow-free data. Our code and dataset are available at: https://github.com/cvlab-stonybrook/ShadowRemovalRefine