Joint Audio-Visual Idling Vehicle Detection with Streamlined Input Dependencies

Xiwen Li, Rehman Mohammed, Tristalee Mangin, Surojit Saha, Kerry Kelly, Ross Whitaker, Tolga Tasdizen; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 885-894

Idling vehicle detection (IVD) can be helpful in monitoring and reducing unnecessary idling and can be integrated into real-time systems to address the resulting pollution and harmful products. The previous approach a non-end-to-end model requires extra user clicks to specify a part of the input making system deployment more error-prone or even not feasible. In contrast we introduce an end-to-end joint audio-visual IVD task designed to detect vehicles visually under three states: moving idling and engine off. Unlike feature co-occurrence task such as audio-visual vehicle tracking our IVD task addresses complementary features where labels cannot be determined by a single modality alone. To this end we propose AVIVD-Net a novel network that integrates audio and visual features through a bidirectional attention mechanism. AVIVD-Net streamlines the input process by learning a joint feature space reducing the deployment complexity of previous methods. Additionally we introduce the AVIVD dataset which is seven times larger than previous datasets offering significantly more annotated samples to study the IVD problem. Our model achieves performance comparable to prior approaches making it suitable for automated deployment. Furthermore by evaluating AVIVDNet on the feature co-occurrence public dataset MAVD we demonstrate its potential for extension to self-driving vehicle video-camera setups.