VideoMem: Constructing, Analyzing, Predicting Short-Term and Long-Term Video Memorability

Romain Cohendet, Claire-Helene Demarty, Ngoc Q. K. Duong, Martin Engilberge; The IEEE International Conference on Computer Vision (ICCV), 2019, pp. 2531-2540

Abstract


Humans share a strong tendency to memorize/forget some of the visual information they encounter. This paper focuses on understanding the intrinsic memorability of visual content. To address this challenge, we introduce a large scale dataset (VideoMem) composed of 10,000 videos with memorability scores. In contrast to previous work on image memorability -- where memorability was measured a few minutes after memorization -- memory performance is measured twice: a few minutes and again 24-72 hours after memorization. Hence, the dataset comes with short-term and long-term memorability annotations. After an in-depth analysis of the dataset, we investigate various deep neural network-based models for the prediction of video memorability. Our best model using a ranking loss achieves a Spearman's rank correlation of 0.494 (respectively 0.256) for short-term (resp. long-term) memorability prediction, while our model with attention mechanism provides insights of what makes a content memorable. The VideoMem dataset with pre-extracted features is publicly available.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Cohendet_2019_ICCV,
author = {Cohendet, Romain and Demarty, Claire-Helene and Duong, Ngoc Q. K. and Engilberge, Martin},
title = {VideoMem: Constructing, Analyzing, Predicting Short-Term and Long-Term Video Memorability},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}