Deep Learning-based Rail Surface Condition Evaluation

Shilin Hu, Ke Ma, Sagnik Das, Dichang Zhang, Dimitris Samaras; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 1366-1375

Rail surface defects can develop into rail cracking, leading to rail failures that pose serious threats to transportation safety. Assessing the severity level of these surface defects is critical for proactive maintenance and long-term asset management. However, existing vision-based inspection systems primarily focus on classifying defect types or locating their positions on the rail surface, offering limited assistance for maintenance planning. To address this gap, we present a unified deep learning framework for automated rail surface defect severity evaluation. Our framework produces direct and interpretable evaluations of the overall surface condition, ranging from level 0 (no defect) to level 7 (severe defect), to support effective maintenance decision-making. The framework comprises three key components: (i) a segmentation module that identifies rail surfaces, eliminating interference from background pixels, (ii) an alignment module that standardizes the pose of rail surfaces to mitigate scale and rotation variance, and (iii) a classification module that predicts defect severity on the aligned rail surfaces. On a new benchmark of expert-labeled high-resolution images, our system achieves 81.8% main-diagonal and 95.9% tri-diagonal accuracy, processing up to 47 images per second. These results demonstrate the reliability and efficiency of our framework for large-scale rail surface monitoring. The dataset and code are available at https://github.com/cvlab-stonybrook/RailEval.