Latency Robust Cooperative Perception using Asynchronous Feature Fusion

Junjie Wang, Tomas Nordström; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 4862-4871

Recent advancements in cooperative perception have showcased substantial improvements compared to single-agent perception. Nonetheless the inherent latency present in such systems can dramatically impair their effectiveness. In this paper we propose a Latency Robust Cooperative Perception framework named LRCP to compensate for the effect of temporal asynchrony. The intuition of LRCP is to directly fuse asynchronous bird's-eye view (BEV) features instead of estimating aligned features. To achieve this we first propose a novel flow prediction module that uses cached past BEV features to predict the flow with a non-discrete time delay at the BEV feature level. Then the predicted flow is employed to guide the spatial sampling location of interests. Our approach substantially enhances the robustness of temporal asynchronous cooperative perception. Specifically we achieved robust performance across a range of latencies up to 500 ms with a performance degradation of only 1 percent point for AP@0.5 metric and 4 percent points for AP@0.7 metric at 500 ms on two public datasets (V2X-Sim and Dair-V2X). Code to reproduce our results is available at https://github. com/JesseWong333/LRCP.