Reconstruct, Rasterize and Backprop: Dense Shape and Pose Estimation From a Single Image

Aniket Pokale, Aditya Aggarwal, Krishna Murthy Jatavallabhula, Madhava Krishna; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 40-41

Abstract


This paper presents a new system to obtain dense object reconstructions along with 6-DoF poses from a single image. Geared towards high fidelity reconstruction, several recent approaches leverage implicit surface representations and deep neural networks to estimate a 3D mesh of an object, given a single image. However, all such approaches recover only the shape of an object; the reconstruction is often in a canonical frame, unsuitable for downstream robotics tasks. To this end, we leverage recent advances in differentiable rendering (in particular, rasterization) to close the loop with 3D reconstruction in camera frame. We demonstrate that our approach---dubbed reconstruct, rasterize and backprop (RRB) achieves significantly lower pose estimation errors compared to prior art, and is able to recover dense object shapes and poses from imagery. We further extend our results to an (offline) setup, where we demonstrate a dense monocular object-centric egomotion estimation system.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Pokale_2020_CVPR_Workshops,
author = {Pokale, Aniket and Aggarwal, Aditya and Jatavallabhula, Krishna Murthy and Krishna, Madhava},
title = {Reconstruct, Rasterize and Backprop: Dense Shape and Pose Estimation From a Single Image},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2020}
}