Recurrent Neural Network for (Un-)Supervised Learning of Monocular Video Visual Odometry and Depth

Rui Wang, Stephen M. Pizer, Jan-Michael Frahm; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5555-5564

Abstract


Deep learning-based, single-view depth estimation methods have recently shown highly promising results. However, such methods ignore one of the most important features for determining depth in the human vision system, which is motion. We propose a learning-based, multi-view dense depth map and odometry estimation method that uses Recurrent Neural Networks (RNN) and trains utilizing multi-view image reprojection and forward-backward flow-consistency losses. Our model can be trained in a supervised or even unsupervised mode. It is designed for depth and visual odometry estimation from video where the input frames are temporally correlated. However, it also generalizes to single-view depth estimation. Our method produces superior results to the state-of-the-art approaches for single-view and multi-view learning-based depth estimation on the KITTI driving dataset.

Related Material


[pdf]
[bibtex]
@InProceedings{Wang_2019_CVPR,
author = {Wang, Rui and Pizer, Stephen M. and Frahm, Jan-Michael},
title = {Recurrent Neural Network for (Un-)Supervised Learning of Monocular Video Visual Odometry and Depth},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}