DeepLocalization: Using Change Point Detection for Temporal Action Localization

Mohammed Shaiqur Rahman, Ibne Farabi Shihab, Lynna Chu, Anuj Sharma; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 7252-7260

In this study we introduce DeepLocalization an innovative framework devised for the real-time localization of actions tailored explicitly for monitoring driver behavior. Utilizing the power of advanced deep learning methodologies our objective is to tackle the critical issue of distracted driving--a significant factor contributing to road accidents. Our strategy employs a dual approach: leveraging Graph-Based Change-Point Detection for pinpointing actions in time alongside a Video Large Language Model (Video-LLM) for precisely categorizing activities. Through careful prompt engineering we customize the Video-LLM to adeptly handle driving activities' nuances ensuring its classification efficacy even with sparse data. Engineered to be lightweight our framework is optimized for consumer-grade GPUs making it vastly applicable in practical scenarios. We subjected our method to rigorous testing on the SynDD2 dataset a complex benchmark for distracted driving behaviors where it demonstrated commendable performance--achieving 57.5% accuracy in event classification and 51% in event detection. These outcomes underscore the substantial promise of DeepLocalization in accurately identifying diverse driver behaviors and their temporal occurrences all within the bounds of limited computational resources.