Probabilistic Structure From Motion With Objects (PSfMO)

Paul Gay, Cosimo Rubino, Vaibhav Bansal, Alessio Del Bue; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 3075-3084

In this paper we deal with the problem of recovering affine camera calibration and objects position/occupancy from multi-view images using the information from image detections. We show that remarkable object localisation and volumetric occupancy can be recovered by including both geometrical constraints and prior information given by objects CAD models from the ShapeNet dataset. This can be done by recasting the problem in the context of a probabilistic framework based on Probabilistic PCA that includes both the object semantic priors together with the multi-view geometrical constraints. We present results on synthetic and real datasets to show the validity of our approach and improvements with respect to previous approaches. In particular, the statistical priors are key to obtain reliable 3D reconstruction especially when the input detections are noisy, a likely case in real scenarios.