How Do Label Errors Affect Thin Crack Detection by DNNs

Liang Xu, Han Zou, Takayuki Okatani; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 4414-4423

Numerous studies have been conducted on detecting cracks in images of roads, surfaces of concrete and other materials, and so on. Accurate annotation of cracks is crucial for supervised learning, but identifying cracks in images, particularly thin cracks, can be challenging, making accurate annotation difficult. However, little is known about how annotation errors in training data affect the accuracy of detectors trained on them. This study attempts to address this gap by synthesizing annotation errors and analyzing their effects, which, to the authors' knowledge, has not been done before in the literature. This is made possible by employing an annotation method that labels cracks as curves with a single-pixel width alongside appropriate training and evaluation methods. We synthesized various types of annotation errors, including under and over-annotation errors caused by crack-like image structures and polyline approximation of crack curves, to reduce annotation costs. The experimental results reveal several important findings, such as that under-annotation is more harmful than over-annotation and that polyline approximation has a modest impact on detection accuracy.