ARTeFACT: Benchmarking Segmentation Models on Diverse Analogue Media Damage

Daniela Ivanova, Marco Aversa, Paul Henderson, John Williamson; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7439-7449

Accurately detecting and classifying damage in analogue media such as paintings photographs textiles mosaics and frescoes is essential for cultural heritage preservation. While machine learning models excel in correcting degradation if the damage operator is known a priori we show that they fail to robustly predict where the damage is even after supervised training; thus reliable damage detection remains a challenge. Motivated by this we introduce ARTeFACT a dataset for damage detection in diverse types analogue media with over 11000 annotations covering 15 kinds of damage across various subjects media and historical provenance. Furthermore we contribute human-verified text prompts describing the semantic contents of the images and derive additional textual descriptions of the annotated damage. We evaluate CNN Transformer diffusion-based segmentation models and foundation vision models in zero-shot supervised unsupervised and text-guided settings revealing their limitations in generalising across media types. Our dataset is available at https://daniela997.github.io/ARTeFACT/ as the first-of-its kind benchmark for analogue media damage detection and restoration.