SynSHRP2: A Synthetic Multimodal Benchmark for Driving Safety-critical Events Derived from Real-world Driving Data

Liang Shi, Boyu Jiang, Zhenyuan Yuan, Miguel A. Perez, Feng Guo; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 4586-4596

Driving-related safety-critical events (SCEs)--including crashes and near-crashes--are crucial for developing and evaluating automated driving systems. However, their rarity and privacy concerns limit data accessibility. The Second Strategic Highway Research Program (SHRP 2) Naturalistic Driving Study (NDS)--the largest NDS to date--captured millions of hours of multimodal, high-frequency driving data, including thousands of SCEs, but public access is restricted. To address this, we introduce SynSHRP2, a synthetic multimodal dataset containing over 1,340 crashes and 5,191 near-crashes derived from SHRP 2 NDS. SynSHRP2 features de-identified keyframes generated via Stable Diffusion and ControlNet, preserving safety-critical details while removing personally identifiable information. It includes SCE type, environmental and traffic annotations, and 10-second kinematic time series before and during each event, alongside synchronized keyframes and narrative descriptions. We present two benchmarks for event attribute classification and scene understanding, highlighting SynSHRP2's potential in advancing safety research and automated driving development. The dataset is available at https://dataverse.vtti.vt.edu/dataset.xhtml?persistentId=doi:10.15787/VTT1/FOZRSM.