Personalized Cinemagraphs Using Semantic Understanding and Collaborative Learning

Tae-Hyun Oh, Kyungdon Joo, Neel Joshi, Baoyuan Wang, In So Kweon, Sing Bing Kang; The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 5160-5169

Abstract


Cinemagraphs are a compelling way to convey dynamic aspects of a scene. In these media, dynamic and still elements are juxtaposed to create an artistic and narrative experience. Creating a high-quality, aesthetically pleasing cinemagraph requires isolating objects in a semantically meaningful way and then selecting good start times and looping periods for those objects to minimize visual artifacts (such a tearing). To achieve this, we present a new technique that uses object recognition and semantic segmentation as part of an optimization method to automatically create cinemagraphs from videos that are both visually appealing and semantically meaningful. Given a scene with multiple objects, there are many cinemagraphs one could create. Our method evaluates these multiple candidates and presents the best one, as determined by a model trained to predict human preferences in a collaborative way. We demonstrate the effectiveness of our approach with multiple results and a user study.

Related Material


[pdf] [Supp]
[bibtex]
@InProceedings{Oh_2017_ICCV,
author = {Oh, Tae-Hyun and Joo, Kyungdon and Joshi, Neel and Wang, Baoyuan and So Kweon, In and Bing Kang, Sing},
title = {Personalized Cinemagraphs Using Semantic Understanding and Collaborative Learning},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}
}