Error Correction for Dense Semantic Image Labeling

Yu-Hui Huang, Xu Jia, Stamatios Georgoulis, Tinne Tuytelaars, Luc Van Gool; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 998-1006

Pixel-wise semantic image labeling is an important, yet challenging task with many applications. Especially in autonomous driving systems, it allows for a full understanding of the system's surroundings, which is crucial for trajectory planning. Typical approaches to tackle this problem involve either the training of deep networks on vast amounts of images to directly infer the labels or the use of probabilistic graphical models to jointly model the dependencies of the input (i.e. images) and output (i.e. labels). Yet, the former approaches do not capture the structure of the output labels, which is crucial for the performance of dense labeling, and the latter rely on carefully hand-designed priors that require costly parameter tuning via optimization techniques, which in turn leads to long inference times. To alleviate these restrictions, we explore how to arrive at dense semantic pixel labels given both the input image and an initial estimate of the output labels. We propose a parallel architecture that: 1) exploits the context information through a LabelPropagation network to propagate correct labels from nearby pixels to improve the object boundaries, 2) uses a LabelReplacement network to directly replace possibly erroneous, initial labels with new ones, and 3) combines the different intermediate results via a Fusion network to obtain the final per-pixel label. We experimentally validate our approach on two different datasets for semantic segmentation, where we show improvements over the state-of-the-art. We also provide both a quantitative and qualitative analysis of the generated results.