Social Aware Multi-Modal Pedestrian Crossing Behavior Prediction

Xiaolin Zhai, Zhengxi Hu, Dingye Yang, Lei Zhou, Jingtai Liu; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 4428-4443

With the development of self-driving vehicles, pedestrian behavior prediction plays a vital role in constructing a safe human-robot interactive environment. Previous methods ignored the inherent uncertainty of pedestrian future actions and the temporal correlations of spatial interactions. To solve the aforementioned problems, we propose a novel social aware multi-modal pedestrian crossing behavior prediction network. In this research field, our network innovatively explores the multimodality nature of pedestrian future action prediction and forecasts diverse and plausible futures. Also, to model the social aware context in both the spatial and temporal domain, we construct a spatial-temporal heterogeneous graph, bridging the spatial-temporal gap between the scene and the pedestrian. Experiments show that our model achieves state-of-the-art performance on pedestrian action detection and prediction task. The code is available at https://github.com/zxll0106/Pedestrian_Crossing_Behavior_Prediction.