Future Video Synthesis With Object Motion Prediction

Yue Wu, Rongrong Gao, Jaesik Park, Qifeng Chen; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 5539-5548

Abstract


We present an approach to predict future video frames given a sequence of continuous video frames in the past. Instead of synthesizing images directly, our approach is designed to understand the complex scene dynamics by decoupling the background scene and moving objects. The appearance of the scene components in the future is predicted by non-rigid deformation of the background and affine transformation of moving objects. The anticipated appearances are combined to create a reasonable video in the future. With this procedure, our method exhibits much less tearing or distortion artifact compared to other approaches. Experimental results on the Cityscapes and KITTI datasets show that our model outperforms the state-of-the-art in terms of visual quality and accuracy.

Related Material


[pdf] [arXiv]
[bibtex]
@InProceedings{Wu_2020_CVPR,
author = {Wu, Yue and Gao, Rongrong and Park, Jaesik and Chen, Qifeng},
title = {Future Video Synthesis With Object Motion Prediction},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}