SinGAN-GIF: Learning a Generative Video Model From a Single GIF

Rajat Arora, Yong Jae Lee; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 1310-1319

Abstract

We propose SinGAN-GIF, an extension of the image based SinGAN to GIFs or short video snippets. Our method learns the distribution of both the image patches in the GIF as well as their motion patterns. We do so by using a pyramid of 3D and 2D convolutional networks to model temporal information while reducing model parameters and training time, along with an image and a video discriminator. SinGAN-GIF can generate similar looking video samples for natural scenes at different spatial resolutions or temporal frame rates, and can be extended to other video applications like video editing, super resolution, and motion transfer. The project page, with supplementary video results, is: https://rajat95.github.io/singan-gif/

Related Material

[pdf]
[bibtex]
@InProceedings{Arora_2021_WACV, author = {Arora, Rajat and Lee, Yong Jae}, title = {SinGAN-GIF: Learning a Generative Video Model From a Single GIF}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2021}, pages = {1310-1319} }