Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo

Francesco Sarno, Suryansh Kumar, Berk Kaya, Zhiwu Huang, Vittorio Ferrari, Luc Van Gool; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 361-371

We present an automated machine learning approach for uncalibrated photometric stereo (PS). Our work aims at discovering a light and computationally efficient PS neural network with excellent surface normal accuracy. Unlike previous uncalibrated deep PS networks, which are handcrafted and carefully tuned, we leverage the recent differentiable neural architecture search (NAS) strategy to find uncalibrated PS architecture automatically. We begin by defining a discrete search space for a light calibration network and a normal estimation network, respectively. We then perform a continuous relaxation of this search space, and present a gradient-based optimization strategy to find an efficient light calibration and normal estimation network. Directly applying the NAS methodology to uncalibrated PS is not straightforward as certain mathematical constraints must be satisfied, which we impose explicitly. Moreover, we search for and train the two networks separately to account for the Generalized Bas Relief (GBR) ambiguity. Extensive experiments on the DiLiGenT benchmark show that the automatically searched neural architectures outperform the current state-of-the-art uncalibrated PS methods, while having a lower memory footprint.