Class Consistent Multi-Modal Fusion With Binary Features

Ashish Shrivastava, Mohammad Rastegari, Sumit Shekhar, Rama Chellappa, Larry S. Davis; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 2282-2291

Abstract


Many existing recognition algorithms combine different modalities based on training accuracy but do not consider the possibility of noise at test time. We describe an algorithm that perturbs test features so that all modalities predict the same class. We enforce this perturbation to be as small as possible via a quadratic program (QP) for continuous features, and a mixed integer program (MIP) for binary features. To efficiently solve the MIP, we provide a greedy algorithm and empirically show that its solution is very close to that of a state-of-the-art MIP solver. We evaluate our algorithm on several datasets and show that the method outperforms existing approaches.

Related Material


[pdf]
[bibtex]
@InProceedings{Shrivastava_2015_CVPR,
author = {Shrivastava, Ashish and Rastegari, Mohammad and Shekhar, Sumit and Chellappa, Rama and Davis, Larry S.},
title = {Class Consistent Multi-Modal Fusion With Binary Features},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2015}
}