Bayesian 3D Tracking from Monocular Video

Ernesto Brau, Jinyan Guan, Kyle Simek, Luca Del Pero, Colin Reimer Dawson, Kobus Barnard; The IEEE International Conference on Computer Vision (ICCV), 2013, pp. 3368-3375


We develop a Bayesian modeling approach for tracking people in 3D from monocular video with unknown cameras. Modeling in 3D provides natural explanations for occlusions and smoothness discontinuities that result from projection, and allows priors on velocity and smoothness to be grounded in physical quantities: meters and seconds vs. pixels and frames. We pose the problem in the context of data association, in which observations are assigned to tracks. A correct application of Bayesian inference to multitarget tracking must address the fact that the model's dimension changes as tracks are added or removed, and thus, posterior densities of different hypotheses are not comparable. We address this by marginalizing out the trajectory parameters so the resulting posterior over data associations has constant dimension. This is made tractable by using (a) Gaussian process priors for smooth trajectories and (b) approximately Gaussian likelihood functions. Our approach provides a principled method for incorporating multiple sources of evidence; we present results using both optical flow and object detector outputs. Results are comparable to recent work on 3D tracking and, unlike others, our method requires no pre-calibrated cameras.

Related Material

author = {Brau, Ernesto and Guan, Jinyan and Simek, Kyle and Del Pero, Luca and Reimer Dawson, Colin and Barnard, Kobus},
title = {Bayesian 3D Tracking from Monocular Video},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {December},
year = {2013}