MDCN-PS: Monocular-Depth-Guided Coarse Normal Attention for Robust Photometric Stereo

Masahiro Yamaguchi, Takashi Shibata, Shoji Yachida, Keiko Yokoyama, Toshinori Hosoi; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 3342-3351

Photometric Stereo (PS) is a technique for estimating surface normals from images illuminated by multiple light sources. However when the target object has a complex shape or the light sources are not appropriately arranged certain regions may experience severe shadows leading to insufficient information for accurate estimation. In this paper we propose a Monocular-Depth-guided Coarse Normal attention for Photometric Stereo (MDCN-PS). The MDCN-PS can effectively combine monocular depth from a single image with PS with multiple light sources by a Photometric Stereo network Adaptor (PS Adaptor) with Coarse Normal Attention. The key is to use the coarse normals obtained from Monocular Depth Estimation as supplementary information which can improve accuracy in regions where the light source is limited due to severe shadows or inhomogeneous light source distribution. Comprehensive experiments on real-world and synthetic datasets show that the proposed method achieved an accuracy improvement of 1.2 points in real-world datasets when limited to two input images and of 3.1 points in synthetic datasets in mean angular error compared to existing methods. Qualitative results also demonstrated that our method improves accuracy in areas with insufficient lighting patterns due to shadows.