A Cross-Season Correspondence Dataset for Robust Semantic Segmentation

Mans Larsson, Erik Stenborg, Lars Hammarstrand, Marc Pollefeys, Torsten Sattler, Fredrik Kahl; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 9532-9542

Abstract


In this paper, we present a method to utilize 2D-2D point matches between images taken during different image conditions to train a convolutional neural network for semantic segmentation. Enforcing label consistency across the matches makes the final segmentation algorithm robust to seasonal changes. We describe how these 2D-2D matches can be generated with little human interaction by geometrically matching points from 3D models built from images. Two cross-season correspondence datasets are created providing 2D-2D matches across seasonal changes as well as from day to night. The datasets are made publicly available to facilitate further research. We show that adding the correspondences as extra supervision during training improves the segmentation performance of the convolutional neural network, making it more robust to seasonal changes and weather conditions.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Larsson_2019_CVPR,
author = {Larsson, Mans and Stenborg, Erik and Hammarstrand, Lars and Pollefeys, Marc and Sattler, Torsten and Kahl, Fredrik},
title = {A Cross-Season Correspondence Dataset for Robust Semantic Segmentation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}