Weighted anomaly scoring for vision-based driving hazard prediction and identification

Nachiket Kamod; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 638-643

With the growing field of autonomous vehicles new obstacles have emerged. Despite developments in artificial intelligence (AI) and sensor technologies which help to assure safety through effective risk assessment the effectiveness of these technologies is still questioned. This work describes a preliminary investigation of a vision-based hazard prediction and identification system employing sequential video frame data. We identify features from the data and use weighted anomaly scoring to estimate risk. The experimental setup evaluates the system's capacity by analyzing the contribution of each feature anomaly to the final risk prediction. This allows us to divide each anomaly into fractions based on its contribution to the final prediction. Furthermore the article calculates the accuracy of open source vision models for recognizing items projected as risks in previous steps. Considered models generate a caption based on the clippings of the predicted hazard and it's surroundings.