3D Motion Decomposition for RGBD Future Dynamic Scene Synthesis

Xiaojuan Qi, Zhengzhe Liu, Qifeng Chen, Jiaya Jia; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7673-7682

Abstract


A future video is the 2D projection of a 3D scene with predicted camera and object motion. Accurate future video prediction inherently requires understanding of 3D motion and geometry of a scene. In this paper, we propose a RGBD scene forecasting model with 3D motion decomposition. We predict ego-motion and foreground motion that are combined to generate a future 3D dynamic scene, which is then projected into a 2D image plane to synthesize future motion, RGB images and depth maps. Optional semantic maps can be integrated. Experimental results on KITTI and Driving datasets show that our model outperforms other state-of-the- arts in forecasting future RGBD dynamic scenes.

Related Material


[pdf]
[bibtex]
@InProceedings{Qi_2019_CVPR,
author = {Qi, Xiaojuan and Liu, Zhengzhe and Chen, Qifeng and Jia, Jiaya},
title = {3D Motion Decomposition for RGBD Future Dynamic Scene Synthesis},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}