CrowdMAC: Masked Crowd Density Completion for Robust Crowd Density Forecasting

Ryo Fujii, Ryo Hachiuma, Hideo Saito; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 8218-8227

A crowd density forecasting task aims to predict how the crowd density map will change in the future from observed past crowd density maps. However the past crowd density maps are often incomplete due to the miss-detection of pedestrians and it is crucial to develop a robust crowd density forecasting model against the miss-detection. This paper presents a MAsked crowd density Completion framework for crowd density forecasting (CrowdMAC) which is simultaneously trained to forecast future crowd density maps from partially masked past crowd density maps (i.e. forecasting maps from past maps with miss-detection) while reconstructing the masked observation maps (i.e. imputing past maps with miss-detection). Additionally we propose Temporal-Density-aware Masking (TDM) which non-uniformly masks tokens in the observed crowd density map considering the sparsity of the crowd density maps and the informativeness of the subsequent frames for the forecasting task. Moreover we introduce multi-task masking to enhance training efficiency. In the experiments CrowdMAC achieves state-of-the-art performance on seven large-scale datasets including SDD ETH-UCY inD JRDB VSCrowd FDST and croHD. We also demonstrate the robustness of the proposed method against both synthetic and realistic miss-detections. The code is released at https://fujiry0.github.io/CrowdMAC-project-page.