Online Adaptive Temporal Memory With Certainty Estimation for Human Trajectory Prediction

Manh Huynh, Gita Alaghband; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp. 940-949

Pedestrian trajectory prediction is an essential component of autonomous systems and robot navigation. Recent research has shown promising predictive performance by designing trajectory prediction networks to model a variety of motion-related features. Different from existing works, our focus is on designing a novel online adaptation framework (OATMem) to exploit the temporal similarities among trajectory samples encountered during testing to improve the prediction accuracy of any such models (i.e., predictors) without knowing the details of these predictors. Our framework consists of two novel modules: an augmented temporal observe-target memory network (ATM) and a certainty-based selector (CS). Inspired by the concept of key-value memory networks [16], we design the ATM to learn the temporal information from short-term past frames by encoding the trajectory samples of past pedestrians in form of observe-target (i.e., key-value) during testing. In addition, we propose a certainty-based selector (CS) to enhance the predictive ability of our framework under scenarios where there are large temporal dissimilarities between current pedestrians' movements and those stored in memory. In dynamic scenes, these scenarios commonly occur due to abrupt changes in contexts, such as camera motions, scene contexts, and pedestrians' behaviors. We extensively evaluate our framework in commonly-used datasets for pedestrian trajectory prediction: JAAD [12] and PIE [19] and show that our framework significantly improves the prediction accuracy of state-of-the-art models. Finally, in-depth ablation studies and analyses are conducted to show on the importance of each proposed component.