Visual Reasoning Using Graph Convolutional Networks for Predicting Pedestrian Crossing Intention

Tina Chen, Renran Tian, Zhengming Ding; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3103-3109

Autonomous vehicles being able to anticipate rather than just react to pedestrian behavior is vital for the harmonious existence of the two on the road. Previous methods for predicting pedestrian crossing intention from the ego-view relied on bounding box location, and if any, limited visual features for their prediction. However, decisions made on the road by drivers and pedestrians are heavily dependent on context, which should be taken into account when trying to predict what pedestrians on the road intend to do. In this paper, we propose using rich visual features in graph convolutional autoencoders to encode the relationship between the pedestrian and its surrounding objects to reason their crossing intention. To further improve prediction results, we also incorporate pedestrian bounding boxes and human pose estimation in the prediction module. Our model differs in that we consider the effects other road objects/agents have on the pedestrian through visual reasoning of those objects/agents. We evaluate our model's performance using balanced accuracy and F1-score to show that we are able to outperform the state-of-the-art. Our model is able to predict crossing intention with 0.79 balanced accuracy, and is able to predict particularly better for cases where the pedestrian has no crossing intention. The code for our model is released at https://github.com/chen289/Visual-GCN.