Silhouette Guided Point Cloud Reconstruction beyond Occlusion

Chuhang Zou, Derek Hoiem; The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 41-50

Abstract


One major challenge in 3D reconstruction is to infer the complete shape geometry from partial foreground occlusions. In this paper, we propose a method to reconstruct the complete 3D shape of an object from a single RGB image, with robustness to occlusion. Given the image and a silhouette of the visible region, our approach completes the silhouette of the occluded region and then generates a point cloud. We show improvements for reconstruction of non-occluded and partially occluded objects by providing the predicted complete silhouette as guidance. We also improve state-of-the-art for 3D shape prediction with a 2D reprojection loss from multiple synthetic views and a surface-based smoothing and refinement step. Experiments demonstrate the efficacy of our approach both quantitatively and qualitatively on synthetic and real scene datasets.

Related Material


[pdf]
[bibtex]
@InProceedings{Zou_2020_WACV,
author = {Zou, Chuhang and Hoiem, Derek},
title = {Silhouette Guided Point Cloud Reconstruction beyond Occlusion},
booktitle = {The IEEE Winter Conference on Applications of Computer Vision (WACV)},
month = {March},
year = {2020}
}