3MASSIV: Multilingual, Multimodal and Multi-Aspect Dataset of Social Media Short Videos

Vikram Gupta, Trisha Mittal, Puneet Mathur, Vaibhav Mishra, Mayank Maheshwari, Aniket Bera, Debdoot Mukherjee, Dinesh Manocha; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 21064-21075

Abstract

We present 3MASSIV, a multilingual, multimodal and multi-aspect, expertly-annotated dataset of diverse short videos extracted from a social media platform. 3MASSIV comprises of 50k short videos (20 seconds average duration) and 100K unlabeled videos in 11 different languages and captures popular short video trends like pranks, fails, romance, comedy expressed via unique audio-visual formats like self-shot videos, reaction videos, lip-synching, self-sung songs, etc. 3MASSIV presents an opportunity for multimodal and multilingual semantic understanding on these unique videos by annotating them for concepts, affective states, media types, and audio language. We present a thorough analysis of 3MASSIV and highlight the variety and unique aspects of our dataset compared to other contemporary popular datasets with strong baselines. We also show how the social media content in 3MASSIV is dynamic and temporal in nature which can be used for various semantic understanding tasks and cross-lingual analysis.

Related Material

[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Gupta_2022_CVPR, author = {Gupta, Vikram and Mittal, Trisha and Mathur, Puneet and Mishra, Vaibhav and Maheshwari, Mayank and Bera, Aniket and Mukherjee, Debdoot and Manocha, Dinesh}, title = {3MASSIV: Multilingual, Multimodal and Multi-Aspect Dataset of Social Media Short Videos}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {21064-21075} }