Evaluating and Bench-marking Object Detection Models for Traffic Sign and Traffic Light Datasets

Ashutosh Mishra, Aman Kumar, Shubham Mandloi, Khushboo Anand, John Zakkam, Seeram Sowmya, Avinash Thakur; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2022, pp. 338-353

Object detection is an important sub-problem for many computer vision applications. There has been substantial research in improving and evaluating object detection models for generic objects but it is still not known how latest deep learning models perform on small road scene objects such as traffic lights and traffic signs. In fact, locating small object of interest such as traffic light and traffic sign is a priority task for an autonomous vehicle to maneuver in complex scenarios. Although some researchers have tried to investigate the performance of deep learning based object detection models on various public datasets, however there exists no comprehensive benchmark. We present a more detailed evaluation by providing in-depth analysis of state-of-theart deep learning based anchor and anchor-less object detection models such as Faster-RCNN, Single Shot Detector (SSD), Yolov3, RetinaNet, CenterNet and Cascade-RCNN. We compare the performance of these models on popular and publicly available traffic light datasets and traffic sign datasets from varied geographies. For traffic light datasets, we consider LISA Traffic Light (TL), Bosch, WPI and recently introduced S2TLD dataset for traffic light detection. For traffic sign benchmarking, we use LISA Traffic Sign (TS), GTSD, TT100K and recently published Mapillary Traffic Sign Dataset (MTSD). We compare the quantitative and qualitative performance of all the models on the aforementioned datasets and find that CenterNet outperforms all other baselines on almost all the datasets. We also compare inference time on specific CPU and GPU versions, flops and parameters for comparison. Understanding such behavior of the models on these datasets can help in solving a variety of practical difficulties and assists in the development of realworld applications. The source code and the models are available at https://github.com/OppoResearchIndia/DLSOD-ACCVW.