Factoring Shape, Pose, and Layout From the 2D Image of a 3D Scene

Shubham Tulsiani, Saurabh Gupta, David F. Fouhey, Alexei A. Efros, Jitendra Malik; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 302-310

Abstract


The goal of this paper is to take a single 2D image of a scene and recover the 3D structure in terms of a small set of factors: a layout representing the enclosing surfaces as well as a set of objects represented in terms of shape and pose. We propose a convolutional neural network-based approach to predict this representation and benchmark it on a large dataset of indoor scenes. Our experiments evaluate a number of practical design questions, demonstrate that we can infer this representation, and quantitatively and qualitatively demonstrate its merits compared to alternate representations.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Tulsiani_2018_CVPR,
author = {Tulsiani, Shubham and Gupta, Saurabh and Fouhey, David F. and Efros, Alexei A. and Malik, Jitendra},
title = {Factoring Shape, Pose, and Layout From the 2D Image of a 3D Scene},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}