Depth Super-Resolution Meets Uncalibrated Photometric Stereo

Songyou Peng, Bjoern Haefner, Yvain Queau, Daniel Cremers; The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2961-2968


A novel depth super-resolution approach for RGB-D sensors is presented. It disambiguates depth super-resolution through high-resolution photometric clues and, symmetrically, it disambiguates uncalibrated photometric stereo through low-resolution depth cues. To this end, an RGB-D sequence is acquired from the same viewing angle, while illuminating the scene from various uncalibrated directions. This sequence is handled by a variational framework which fits high-resolution shape and reflectance, as well as lighting, to both the low-resolution depth measurements and the high-resolution RGB ones. The key novelty consists in a new PDE-based photometric stereo regularizer which implicitly ensures surface regularity. This allows to carry out depth super-resolution in a purely data-driven manner, without the need for any ad-hoc prior or material calibration. Real-world experiments are carried out using an out-of-the-box RGB-D sensor and a hand-held LED light source.

Related Material

[pdf] [arXiv]
author = {Peng, Songyou and Haefner, Bjoern and Queau, Yvain and Cremers, Daniel},
title = {Depth Super-Resolution Meets Uncalibrated Photometric Stereo},
booktitle = {The IEEE International Conference on Computer Vision (ICCV) Workshops},
month = {Oct},
year = {2017}