Siamese CNNs for RGB-LWIR Disparity Estimation

David-Alexandre Beaupre, Guillaume-Alexandre Bilodeau; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2019, pp. 0-0

Currently, for the task of color (RGB) and thermal in- frared (LWIR) disparity estimation, handcrafted feature descriptors such as mutual information are the methods achieving best performance. In this work, we aim to as- sess if convolutional neural networks (CNNs) can achieve competitive performance in this task. We developed an ar- chitecture made of two subnetworks, each consisting of the same siamese network, but taking different image patches as input. Each siamese network, in the feature space, searches for the disparity between the left and right patch. The out- put of the two subnetworks are summed together so that we can be more confident in the predicted disparity by enforc- ing left-right consistency. We show that having two subnet- works working together in parallel to get the final prediction helps achieve better performance when compared to a sin- gle subnetwork by itself. We tested our method on the LITIV dataset and found the results competitive when compared to handcrafted feature descriptors. The source code of our method will be available online upon publication.