Semiotic-Based Construction of a Large Emotional Image Dataset with Neutral Samples

Marco Blanchini, Giovanna Dimitri, Lydia Abady, Benedetta Tondi, Tarcisio Lancioni, Mauro Barni; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7541-7550

Image Visual Sentiment Analysis (VSA) requires the availability of large annotated datasets whose construction presents many challenges. The necessity of gathering a large amount of labeled images contrasts with the rigorous but lengthy process required for manual annotation based on psychovisual experiments and with the automatic gathering of large amounts of data roughly labeled based on the sentiment analysis of the text accompanying the images like captions tweets and tags. An additional limitation is the scarcity of high-quality datasets with a neutral class which forces the images to be classified into emotions even when the observers show no emotional activation. In this work we present a scalable methodology rooted in semiotics and art theory for the construction of a 3-class (positive negative and neutral) VSA dataset enabling the downloading of a desired quantity of images while maintaining labeling coherence and accuracy. Based on the proposed methodology we introduce and make publicly available a VSA dataset of over 100000 images. To validate the quality of the dataset we used it to train several classifiers and compared their performance with those of classifiers trained on other datasets. The results show that the classifiers trained on the new dataset provide better performance when tested on independent datasets including those commonly used for psychovisual experiments.