Scaling CNNs for High Resolution Volumetric Reconstruction From a Single Image

Adrian Johnston, Ravi Garg, Gustavo Carneiro, Ian Reid, Anton van den Hengel; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 939-948

Abstract


One of the long-standing tasks in computer vision is to use a single 2-D view of an object in order to produce its 3-D shape. Recovering the lost dimension in this process has been the goal of classic shape-from-X methods, but often the assumptions made in those works are quite limiting to be useful for general 3-D objects. This problem has been recently addressed with deep learning methods containing a 2-D (convolution) encoder followed by a 3-D (deconvolution) decoder. These methods have been reasonably successful, but memory and run time constraints impose a strong limitation in terms of the resolution of the reconstructed 3-D shapes. In particular, state-of-the-art methods are able to reconstruct 3-D shapes represented by volumes of at most 32^3 voxels using state-of-the-art desktop computers. In this work, we present a scalable 2-D single view to 3-D volume reconstruction deep learning method, where the 3-D (deconvolution) decoder is replaced by a simple inverse discrete cosine transform (IDCT) decoder. Our simpler architecture has an order of magnitude faster inference when reconstructing 3-D volumes compared to the convolution-deconvolutional model, an exponentially smaller memory complexity while training and testing, and a sub-linear runtime training complexity with respect to the output volume size. We show on benchmark datasets that our method can produce high-resolution reconstructions with state of the art accuracy.

Related Material


[pdf]
[bibtex]
@InProceedings{Johnston_2017_ICCV,
author = {Johnston, Adrian and Garg, Ravi and Carneiro, Gustavo and Reid, Ian and van den Hengel, Anton},
title = {Scaling CNNs for High Resolution Volumetric Reconstruction From a Single Image},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops},
month = {Oct},
year = {2017}
}