Temporal Dynamics in Visual Data: Analyzing the Impact of Time on Classification Accuracy

Tom Pégeot, Eva Feillet, Adrian Popescu, Inna Kucher, Bertrand Delezoide; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 6932-6943

Visual datasets are generally constructed from the samples available at the time of their collection and are not further updated. However these static datasets do not reflect the distribution changes that occur in real data. We analyze how different collection times lead to a shift in class distribution by collecting a set of Flickr images published over 14 years. The proposed "Visual Classes through Time" (VCT-107) dataset contains images tagged by their publication date and includes 107 classes covering various topics (human-made objects animals plants food etc.). Images from each class are divided into five collection periods to study the impact of time on classification accuracy. When training different classification models using linear probing we observe an accuracy loss when training on data from one period and testing on other periods. This happens even in the case of a strongly pre-trained model like DinoV2 ViT-B/14. Intuitively the performance loss is generally more significant when the collection periods between the training and test data are further apart. Our analysis reveals that the temporal shift varies between classes with the largest shifts observed for human-made objects and the smallest for natural concepts such as animal species. Our results stress the importance of regularly updating models to adapt to time-induced changes in the distribution of visual classes even when using a strongly pre-trained model. We release the VCT-107 dataset to facilitate research on temporal shifts.