Mutual Foreground Segmentation With Multispectral Stereo Pairs

Pierre-Luc St-Charles, Guillaume-Alexandre Bilodeau, Robert Bergevin; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 375-384


Foreground-background segmentation of video sequences is a low-level process commonly used in machine vision, and highly valued in video content analysis and smart surveillance applications. Its efficacy relies on the contrast between objects observed by the sensor. In this work, we study how the combination of sensors operating in the long-wavelength infrared (LWIR) and visible spectra can improve the performance of foreground-background segmentation methods. As opposed to a classic visible spectrum stereo pair, this multispectral pair is more adequate for object segmentation since it reduces the odds of observing low-contrast regions simultaneously in both images. We show that by alternately minimizing stereo disparity and binary segmentation energies with dynamic priors, we can drastically improve the results of a traditional video segmentation approach applied to each sensor individually. Our implementation is freely available online for anyone wishing to recreate our results.

Related Material

author = {St-Charles, Pierre-Luc and Bilodeau, Guillaume-Alexandre and Bergevin, Robert},
title = {Mutual Foreground Segmentation With Multispectral Stereo Pairs},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops},
month = {Oct},
year = {2017}